Andrew Saxe

Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures? Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham arxiv.org/abs/2512.20607

Looking for alternatives to quadratic functions for closed-form analysis in optimization? This post explores matrix Riccati dynamics and their applications to neural networks. francisbach.com/closed-form-dy…

What is the relationship between memorization and generalization in AI? Is there a fundamental tradeoff? In a new blog post I’ve reviewed some of the evolving perspectives on memorization & generalization in machine learning, from classic perspectives through LLMs. Link below:

📢 We’re now accepting applications for the 2026 School on Analytical Connectionism dedicated this year to Language Acquisition. 📍 Gothenburg, Sweden 🗓️ August 17–28, 2026 ☠️ Apply by April 17! 🔗 analytical-connectionism.net/school/2026/ 👇 Meet the experts joining us this summer!

Hiring 2 Postdocs to work on Theoretical Foundations of AI Safety @chalmersuniv If you have a background in Physics, Math, or ML and want to tackle AI alignment at a fundamental level alongside UCL, apply below! 🔗Apply: chalmers.se/en/about-chalm… 🔬Lab: stefsmlab.github.io

How to apply: USD 80,000–100,000 (50-74k GBP) annualized 6 months, w/ extension based on funding Details: docs.google.com/document/d/1Vj… Application: forms.gle/xKukH74iX16pPG… 4

We’re hiring postdocs/research scientists! Your interests can be anywhere on the spectrum from pure theory to empirically testing predictions relevant to AI safety. Our theoretical work relies on dynamical systems and tools from statistical physics. 3

Excited to launch Principia, a nonprofit research organisation at the intersection of deep learning theory and AI safety. Our goal is to develop theory for modern machine learning that can help us understand network behaviors, including those critical for AI safety. 1

Upcoming online talk next Monday 9th February, at the ELLIS Reading Group on Mathematics & Efficiency of Deep Learning! Open to all. Info at sites.google.com/view/efficient…

Equipped with this theory, we make new predictions about how network width, data distribution, and initialization affect learning dynamics. For example, increasing the number of attention heads in linear attention shortens the plateaus in learning.

Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures? Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham arxiv.org/abs/2512.20607

📢 Applications open on 19 Jan for 7-week #Mathematics #SummerSchool in London. You will develop maths skills and intuition necessary to enter #TheoreticalNeuroscience or #MachineLearning field. ℹ️ Find out more & register for information webinar 👉 ucl.ac.uk/life-sciences/…

📢 Job alert We are looking for a Postdoctoral Fellow to work with @ArthurGretton on creating statistically efficient causal and interaction models with the aim of elucidating cellular interactions. ⏰Deadline 27-Aug-2025 ℹ️ ucl.ac.uk/work-at-ucl/se…

🎓Thrilled to share I’ve officially defended my PhD!🥳 At @GatsbyUCL, my research explored how prior knowledge shapes neural representations. I’m deeply grateful to my mentors, @SaxeLab and Caswell Barry, my incredible collaborators, and everyone who supported me! Stay tuned!