12.11.2026 14:00 Giuseppe Bruno (U Bern):
Scaling limits in mean-field transformer modelsOnline: attend (Meeting ID: 815 6386 5065 Passcode: 929409 )02.07.014 (Boltzmannstr. 3, 85748 Garching)

In this talk, I will present two results on the dynamics of token representations in Transformer models, using tools from probability and PDE analysis. In the first part, I will discuss the stochastic scaling limit of deep and wide Transformers, and show how noise can lead to synchronization between tokens. In the second part, I will study a class of nonlocal transport equations arising from self-attention and show that their solutions converge to the heat equation, with quantitative estimates that are uniform in time. Together, these results provide a mathematical description of how interactions between tokens can lead to different collective behaviors in deep Transformers.