Seminars

The first seminar of the series, on one of the most interesting topics in AI safety research: mechanistic interpretability.

When a large language model refuses a request, what actually happens inside it? Can we identify the mechanisms behind this behaviour and intervene on them directly? Through a practical example on refusal behaviour, we will see how mechanistic interpretability can help us understand how LLMs work inside, and change how they behave.

How can we tell whether a text was written by a language model? One answer is to have the model sign what it writes: a watermark hidden in its choice of words, invisible to readers but detectable by anyone who holds the key.

Starting from the paper that proposed it, we will see how the technique works and where its limits are. An entry-level talk: no background needed. For a preview, Carlo's post on the topic (in Italian) is on his Substack.

What do Transformer models actually perceive? Most interpretability methods assume that the input space is flat and Euclidean, but networks deform their representation spaces during training: inputs that look far apart can be the same to the model, while small changes can flip its output.

The talk presents a framework based on Riemannian geometry to find the sets of inputs a model treats as equivalent, and to move within them and across their boundaries. It then argues that the right language for understanding neural networks may be differential geometry rather than linear algebra.