Jianliang He

New Paper -- "On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking" We give a complete mechanistic and dynamic picture of how neural networks learn modular addition f(x,y) = (x+y) mod p. We answer three questions: (1) What does the trained network compute? (2) How do Fourier features emerge during training? (3) Why does grokking happen? Each answer comes with a mathematical characterization backed by theory and experiments. Paper: arxiv.org/abs/2602.16849 Blog: y-agent.github.io/posts/modular_… Demo: huggingface.co/spaces/y-agent… Code: github.com/Y-Agent/modula…

[New paper on in-context learning] "In-Context Linear Regression Demystified" (link: arxiv.org/abs/2503.12734). Joint work @JLiangHe, @xintianpan, @siyuc3141. We establish a rather complete understanding of how one-layer multi-head attention solves in-context linear regression,