Pascal Tikeng (migrant)

Reviewers & ACs for #ICML2026 have been recognized for their service! - Reviewers: 4439 Gold (free registration), 4437 Silver. 17749 total reviewers were assigned >= 1 paper - ACs: 1647 receive free registration, out of 1691 who were assigned >= 1 paper TY for your hard work!

Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok Pascal Jr. Tikeng Notsawo , Hattie Zhou (@oh_that_hat), Mohammad Pezeshki (@mpezeshki91), Irina Rish (@irinarish), Guillaume Dumas (@introspection) arxiv.org/abs/2306.13253 Tags: grokking, science of DL, mechanistic interpretability, Mila/Meta If you want to know if your model will eventually grok without training it for a long time, the authors suggest looking at the loss during early epochs. In brief, low-frequency oscillations of the loss can be a good proxy for upcoming grokking. The authors contextualize this with other findings of grokking (specifically the sling-shot mechanism) and loss landscape research. I think the biggest appeal this paper had for me was how this research fits into the larger "science of DL" research landscape, and I will probably revisit it because it is so helpful to see how a lot of these concepts agree or disagree.