In this talk, I will present two results on the dynamics of token representations in Transformer models, using tools from probability and PDE analysis. In the first part, I will discuss the stochastic scaling limit of deep and wide Transformers, and show how noise can lead to synchronization between tokens. In the second part, I will study a class of nonlocal transport equations arising from self-attention and show that their solutions converge to the heat equation, with quantitative estimates that are uniform in time. Together, these results provide a mathematical description of how interactions between tokens can lead to different collective behaviors in deep Transformers.