Andrew Saxe

745 posts

Andrew Saxe banner
Andrew Saxe

Andrew Saxe

@SaxeLab

Prof at @GatsbyUCL and @SWC_Neuro, trying to figure out how we learn. Bluesky: @SaxeLab Mastodon: @[email protected]

London, UK Katılım Kasım 2019
379 Takip Edilen6K Takipçiler
Sabitlenmiş Tweet
Andrew Saxe
Andrew Saxe@SaxeLab·
Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures? Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham arxiv.org/abs/2512.20607
GIF
English
10
58
439
29.8K
Andrew Saxe retweetledi
Azwar Abdulsalam
Azwar Abdulsalam@Azlock1729·
New paper w/ @SaxeLab & Nishil Patel! 🧵 Does RL post-training teach models anything new, or just amplify skills already in the base model? We built a fully auditable testbed to settle it — and caught RL composing new strategies in the act.
English
5
5
52
6.5K
Andrew Saxe retweetledi
Zixuan Wang
Zixuan Wang@zzZixuanWang·
A question on synthetic data generation: If we want a language model to solve k-step arithmetic problems (such as a+b*c-d=?), with operands from 1 to 100, which training distribution should we use? A. Uniform distribution: Sample these k operands uniformly from 1 to 100 B. Power law: randomly shuffle 1-100 and impose an artificial power law. Sample these k operands according to this power law. ⚡Our ICML 2026 (spotlight) paper shows: Option B is better! Surprisingly, the same idea extends far beyond this simple example to many reasoning tasks that require implicit composition of multiple atomic skills, including multi-hop QAs and synthetic GSM problems. 📄Paper: arxiv.org/abs/2604.22951 📝Blog: zixuan-wang-dlt.github.io/posts/2026/06/…
Zixuan Wang tweet media
English
13
56
220
44.1K
Andrew Saxe retweetledi
Clémentine Dominé, Phd 🍊
Clémentine Dominé, Phd 🍊@ClementineDomi6·
Pretraining + fine-tuning powers modern ML, but we lack a theoretical understanding of how pretraining actually shapes downstream learning. Enter our new @icmlconf paper: “A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning” 📅 July 9th, Poster #4502 Session 8! 🧵
Clémentine Dominé, Phd 🍊 tweet media
English
1
14
86
6.8K
Andrew Saxe retweetledi
Stefano Sarao Mannelli
Stefano Sarao Mannelli@stefsmlab·
Interested in any of our works? Come and say hi @icmlconf! Unfortunately I won't be around this year but my collaborators will answer all your questions :)
Cognition, Adaptation and Learning (CAandL) Lab@caandl_lab

We are excited to share that CAandL Lab will feature in 3 papers at #ICML2026 this week. Stop by and say hi if you are interested in any of these! @devonjarvi5, @stefsmlab, @geraudnt with @kleinric, @BenjaminRosman, Damien Harvey, Branden Ingram, and Steven James

English
1
2
8
1.3K
Andrew Saxe retweetledi
Sara Dragutinovic
Sara Dragutinovic@sara_drag·
Is Muon as good as they say? We looked beyond training speed and found a hidden cost: Muon loses the simplicity bias of older optimizers like gradient descent — and this matters for generalization.
Sara Dragutinovic tweet media
English
14
41
365
37.1K
Andrew Saxe retweetledi
David Barber
David Barber@davidobarber·
Very proud to be leading the SOFAIR Lab ucl.ac.uk/engineering/so… with fantastic colleagues from UCL, Edinburgh, Oxford and Cambridge.
Kanishka Narayan MP@KanishkaNarayan

🚨 2 major new AI labs in the UK 🚨 3 months ago, we launched a £40m call for a fundamental AI lab in the UK. Given the exceptional bids we received, we have doubled down: 2 new AI labs, £60m seed funding. ▪️@BOLD_LAB_AI: led by @j_foerst, with an exceptional team of @CULLYAntoine, @shimon8282, @tonizza82, Ani Calinescu & @_rockt ▪️SOFAIR: led by Prof David Barber, with a world-leading team of Mirella Lapata, @yaringal and @LourdesAgapito The best of UK academia, government and industry, together, to make fundamental advances in AI 🚀 researchprofessionalnews.com/rr-news-uk-pol…

English
2
16
58
9.7K
Andrew Saxe retweetledi
Yoshua Bengio
Yoshua Bengio@Yoshua_Bengio·
Europe has a lot to lose in the current AI race, and it's worth examining how threats to middle-power sovereignty can result in unsafe outcomes. Such scenarios help illustrate why Europe must invest in AI initiatives that can either leapfrog the current frontier or offer critical components like safety and reliability.
Alex Petropoulos 🤠@AlexTPet

I'm deeply concerned about Europe's future on AI. One of my biggest worries is our erosion of agency, our ability to stay relevant and fight for our values in a future where AI becomes a civilisationally important technology. Myself, @DadaJudith , @bakkermichiel and others have written a scenario to outline a potential future we worry we are on track towards. europe2031.ai Every optimistic and realistic path I can see for Europe runs through a central node - one where Europe has more leverage, more importance and more say. One where Europe grows more, builds more where it matters, and takes ownership over its resilience. Europe 2031 is a five-year scenario of the continent's slide into irrelevance: how AI is driving it, and what can still be done. The co-authors are researchers, scientists and investors who have advised European leaders, co-authored national AI strategies, built and funded these systems from the inside. We have no interest in hype and we deeply care about this continent. Europe 2031 ends with five concrete recommendations: - drastically more compute on European soil - an AI middle-power coalition - labour-market reforms - a bold position in robotics and industrial AI - and a positive vision of what AI can do for society. Europe can still change course if it finds the political will and the courage to engage in the most ambitious political and economic agenda the continent has undertaken in peacetime. I encourage you to read it if you have the time:

English
13
30
190
35.9K
Andrew Saxe retweetledi
Stefano Sarao Mannelli
Stefano Sarao Mannelli@stefsmlab·
Model collapse is often framed as “models getting worse” In our ICML Spotlight Position paper, we show a high risk of unequal degradation. Rare languages, minority viewpoints, and low-resource communities are likely to be affected first and most severely arxiv.org/abs/2605.04127
Devon Jarvis@devonjarvi5

I'm excited to share our position paper that has been accepted at ICML as a Spotlight paper. In this work we (@kleinric, @BenjaminRosman, Steven James and @stefsmlab) make a call to action for more focus on model collapse in the AI Fairness community arxiv.org/pdf/2605.04127

English
0
1
22
3.3K
Andrew Saxe retweetledi
Andrew Saxe retweetledi
Jamie Simon
Jamie Simon@learning_mech·
1/ Deep learning is going to have a scientific theory. We can see the pieces starting to come together, and it's looking a lot like physics! We're releasing a paper pulling together these emerging threads and giving them a name: learning mechanics. 🔨 arxiv.org/pdf/2604.21691 🔧
Jamie Simon tweet media
English
54
290
1.5K
306.8K
Andrew Saxe retweetledi
Stefano Sarao Mannelli
Stefano Sarao Mannelli@stefsmlab·
Two Analytical Connectionism-related updates: 1. ⏰ 1 week left to apply! Interested in language + AI & cognition? Don’t miss it: analytical-connectionism.net/school/2026/ 2. 📜 Lecture notes from the first two editions are finally out: proceedings.mlr.press/v320/
Stefano Sarao Mannelli@stefsmlab

📢 We’re now accepting applications for the 2026 School on Analytical Connectionism dedicated this year to Language Acquisition. 📍 Gothenburg, Sweden 🗓️ August 17–28, 2026 ☠️ Apply by April 17! 🔗 analytical-connectionism.net/school/2026/ 👇 Meet the experts joining us this summer!

English
0
5
14
2.6K
Andrew Saxe
Andrew Saxe@SaxeLab·
Postdoc opening! Come work with us on deep learning theory relevant to AI safety Deadline: 7 Apr 2026 Details and application: ucl.ac.uk/work-at-ucl/se…+
English
0
36
129
11.9K
Andrew Saxe
Andrew Saxe@SaxeLab·
Very excited by this year's Analytical Connectionism Summer School! A dream lineup of speakers on the topic of language acquisition in minds and machines Bursaries available to cover costs Aug 17 – Aug 28, 2026 Gothenburg Details: analytical-connectionism.net//school/2026/
Andrew Saxe tweet media
English
0
6
34
3.3K
Andrew Saxe retweetledi
Francis Bach
Francis Bach@BachFrancis·
Looking for alternatives to quadratic functions for closed-form analysis in optimization? This post explores matrix Riccati dynamics and their applications to neural networks. francisbach.com/closed-form-dy…
GIF
English
0
23
162
9.5K
Andrew Saxe retweetledi
Andrew Lampinen
Andrew Lampinen@AndrewLampinen·
What is the relationship between memorization and generalization in AI? Is there a fundamental tradeoff? In a new blog post I’ve reviewed some of the evolving perspectives on memorization & generalization in machine learning, from classic perspectives through LLMs. Link below:
Andrew Lampinen tweet media
English
12
43
424
23.8K
Andrew Saxe retweetledi
Stefano Sarao Mannelli
Stefano Sarao Mannelli@stefsmlab·
📢 We’re now accepting applications for the 2026 School on Analytical Connectionism dedicated this year to Language Acquisition. 📍 Gothenburg, Sweden 🗓️ August 17–28, 2026 ☠️ Apply by April 17! 🔗 analytical-connectionism.net/school/2026/ 👇 Meet the experts joining us this summer!
Stefano Sarao Mannelli tweet media
English
1
15
42
10.3K
Andrew Saxe
Andrew Saxe@SaxeLab·
We’re hiring postdocs/research scientists! Your interests can be anywhere on the spectrum from pure theory to empirically testing predictions relevant to AI safety. Our theoretical work relies on dynamical systems and tools from statistical physics. 3
English
2
2
48
3K
Andrew Saxe
Andrew Saxe@SaxeLab·
Excited to launch Principia, a nonprofit research organisation at the intersection of deep learning theory and AI safety. Our goal is to develop theory for modern machine learning that can help us understand network behaviors, including those critical for AI safety. 1
English
9
35
302
19.1K