Anders Larsen

Really happy that our paper on solving the crystallographic phase problem using neural networks is out in @ScienceMagazine. Thanks to @mol_crystal_guy and @AndersOMadsen for a great collaboration. science.org/doi/10.1126/sc…

Unveiling our new startup Advanced Machine Intelligence (AMI Labs). We just completed our seed round: $1.03B / 890M€, one the largest seeds ever, probably the largest for a European company. We're hiring! [the background image is the Veil Nebula - a picture I took from my backyard, most appropriate for an unveiling] More details here: techcrunch.com/2026/03/09/yan…

FlexAttention now has a FlashAttention-4 backend. FlexAttention has enabled researchers to rapidly prototype custom attention variants—with 1000+ repos adopting it and dozens of papers citing it. But users consistently hit a performance ceiling. Until now. We've added a FlashAttention-4 backend to FlexAttention on Hopper and Blackwell GPUs. PyTorch now auto-generates CuTeDSL score/mask modifications and JIT-instantiates FlashAttention-4 for your custom attention variant. The result: 1.2× to 3.2× speedups over Triton on compute-bound workloads. 🖇️ Read our latest blog here: hubs.la/Q045FHPh0 No more choosing between flexibility and performance. hashtag#PyTorch hashtag#FlexAttention hashtag#FlashAttention hashtag#OpenSourceAI

We have a new preprint out on OpenFF's new force field and new, much faster charge model: chemrxiv.org/engage/chemrxi…

New paper 📜: Tiny Recursion Model (TRM) is a recursive reasoning approach with a tiny 7M parameters neural network that obtains 45% on ARC-AGI-1 and 8% on ARC-AGI-2, beating most LLMs. Blog: alexiajm.github.io/2025/09/29/tin… Code: github.com/SamsungSAILMon… Paper: arxiv.org/abs/2510.04871

Scalable emulation of protein equilibrium ensembles with generative deep learning => now out in Science Magazine! science.org/doi/10.1126/sc…

🚀 After two+ years of intense research, we’re thrilled to introduce Skala — a scalable deep learning density functional that hits chemical accuracy on atomization energies and matches hybrid-level accuracy on main group chemistry — all at the cost of semi-local DFT. ⚛️🔥🧪🧬

Join us to work on LLMs for drug discovery, including scaling/optimizing large model training and inference workflows on our cutting-edge infrastructure, pre-training, post-training, and multimodal learning and integrating non-text modalities. apply.deshawresearch.com/careers/Regist…

btw, i wrote a post about "how to scale" based on what i've learned over the past few months. it covers muP, HP scaling laws, and some stuffs. would be happy to get any feedback or discussion. (it's pretty verbose and no TL;DR, sorry lol) howtoscalenn.github.io

New blog post: let's talk about latents! sander.ai/2025/04/15/lat…

New paper - Transformers, but without normalization layers (1/n)

A thread on our new paper Thermodynamic Bayesian Inference 250 years later, Bayes’s theorem is still the gold standard for probabilistic reasoning. But for complicated models it’s too hard to implement exactly, so approximations are used. For example, the complexity of Bayesian Neural Network posteriors makes them hard to sample from (see cims.nyu.edu/~andrewgw/bnnh…).

@cloneofsimo Can you use the same learning rate and weight decay as with Adam or do you have to do a new search?

Shampoo Scaling law for language model Plot taste of Kaplan et al, but comparing shampoo and adam. Shampoo is literally such a free lunch, in large scale, in predictable manner.

Data generation through simulation is one reason why the whole idea of ML for science is so exciting.

For too long, users have lived under the software lottery tyranny of fused attention implementations. No longer. Introducing FlexAttention, a new PyTorch API allowing for many attention variants to enjoy fused kernels in a few lines of PyTorch. pytorch.org/blog/flexatten… 1/10

Our neural network, PhAI, accurately predicts phases from amplitudes without assumptions about crystal contents. No prior knowledge needed and PhAI does very well with low-resolution data (e.g., 2.0 Å)

Our research reveals neural networks could improve structure solutions for weakly scattering crystals! This includes protein crystals, metal-organic frameworks, and nanometer-sized crystals often seen in electron diffraction.

See this perspective for more information science.org/content/articl…

Really happy that our paper on solving the crystallographic phase problem using neural networks is out in @ScienceMagazine. Thanks to @mol_crystal_guy and @AndersOMadsen for a great collaboration. science.org/doi/10.1126/sc…

This is exactly what I hate with all big frameworks. TF is terrible. PyTorch used to be straightforward but turned terrible too. Torch7 was very direct. JAX/Flax still ok, but I pray every day that it doesn’t end up with the same fate over time.

Tried to capture why I’m so excited about NNPs. I honestly think they are a profoundly useful tool.