Andrew Saxe

Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures? Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham arxiv.org/abs/2512.20607

1/ Deep learning is going to have a scientific theory. We can see the pieces starting to come together, and it's looking a lot like physics! We're releasing a paper pulling together these emerging threads and giving them a name: learning mechanics. 🔨 arxiv.org/pdf/2604.21691 🔧

Come chat about this @iclr_conf, at 3:15 PM on Friday in Pavilion 4 Poster #4216!

Two Analytical Connectionism-related updates: 1. ⏰ 1 week left to apply! Interested in language + AI & cognition? Don’t miss it: analytical-connectionism.net/school/2026/ 2. 📜 Lecture notes from the first two editions are finally out: proceedings.mlr.press/v320/

Postdoc opening! Come work with us on deep learning theory relevant to AI safety Deadline: 7 Apr 2026 Details and application: ucl.ac.uk/work-at-ucl/se…+

Very excited by this year's Analytical Connectionism Summer School! A dream lineup of speakers on the topic of language acquisition in minds and machines Bursaries available to cover costs Aug 17 – Aug 28, 2026 Gothenburg Details: analytical-connectionism.net//school/2026/

Looking for alternatives to quadratic functions for closed-form analysis in optimization? This post explores matrix Riccati dynamics and their applications to neural networks. francisbach.com/closed-form-dy…

What is the relationship between memorization and generalization in AI? Is there a fundamental tradeoff? In a new blog post I’ve reviewed some of the evolving perspectives on memorization & generalization in machine learning, from classic perspectives through LLMs. Link below:

📢 We’re now accepting applications for the 2026 School on Analytical Connectionism dedicated this year to Language Acquisition. 📍 Gothenburg, Sweden 🗓️ August 17–28, 2026 ☠️ Apply by April 17! 🔗 analytical-connectionism.net/school/2026/ 👇 Meet the experts joining us this summer!

Hiring 2 Postdocs to work on Theoretical Foundations of AI Safety @chalmersuniv If you have a background in Physics, Math, or ML and want to tackle AI alignment at a fundamental level alongside UCL, apply below! 🔗Apply: chalmers.se/en/about-chalm… 🔬Lab: stefsmlab.github.io

How to apply: USD 80,000–100,000 (50-74k GBP) annualized 6 months, w/ extension based on funding Details: docs.google.com/document/d/1Vj… Application: forms.gle/xKukH74iX16pPG… 4

We’re hiring postdocs/research scientists! Your interests can be anywhere on the spectrum from pure theory to empirically testing predictions relevant to AI safety. Our theoretical work relies on dynamical systems and tools from statistical physics. 3

Excited to launch Principia, a nonprofit research organisation at the intersection of deep learning theory and AI safety. Our goal is to develop theory for modern machine learning that can help us understand network behaviors, including those critical for AI safety. 1

Upcoming online talk next Monday 9th February, at the ELLIS Reading Group on Mathematics & Efficiency of Deep Learning! Open to all. Info at sites.google.com/view/efficient…

Equipped with this theory, we make new predictions about how network width, data distribution, and initialization affect learning dynamics. For example, increasing the number of attention heads in linear attention shortens the plateaus in learning.

Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures? Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham arxiv.org/abs/2512.20607