Gabe Guo 🦄

🚨New time series generative model just dropped. Paper: arxiv.org/abs/2604.27443 Demo: abc-diffusion.github.io ⏰Meet ABC: Any-Subset Autoregressive Diffusion Bridges in Continuous Time & Space. With @StefanoErmon @elon_lit @Jose_Blanchet @thanawatsornwan @lutong_hao

Fixed point iterations for parallelizing nonlinear dynamics is all the rage: - Newton for RNNs - Picard for diffusion models - Jacobi for parallel decode of LLMs But how do these techniques relate, and when should you use them? We show you how in our new paper 🧵

I am thrilled to share a paradigm-changing work in generative modeling: Flux Matching by the very brilliant graduate student Peter @peterpaohuang (co-mentored with @StefanoErmon). By extending beyond the score functions used in diffusion models to a broader class of vector fields, Flux Matching enables structural priors in dynamics, faster sampling, more interpretable generation, and many new possibilities. In biology, Peter shows that replacing the EM algorithm in scVelo with Flux Matching can dramatically improve RNA velocity accuracy, including cross-boundary correctness and consistency. Its ability to train on large-scale single-cell and perturbation data makes it especially exciting for building better causal virtual cell and virtual embryo models. I am deeply grateful for the support from Laude Institute @LaudeInstitute , Pantas and Ting Sutardja Foundation, the Wu Tsai Neurosciences Institute Big Ideas in Neuroscience Program, NIH DP2 grant 1DP2OD037052-01, and NIH K99/R00 grant 4K99HG012887-02 @NIH_CommonFund. Most importantly, I am deeply honored to have Peter as the first graduate student in the lab! I want to congratulate Peter on this outstanding achievement. He developed this idea independently, drawing on his background in causal learning, diffusion models, and Perturb-seq, and pushed through many technical challenges with remarkable creativity, persistence, and diligence. I cannot wait to see the impact this work will have in both machine learning and biology! See more information from Peter below:

Introducing Flux Matching, a generative modeling paradigm that generalizes diffusion models to vector fields that need not be the score function. Enables structural priors in the dynamics, faster sampling, interpretable generation, and more! w/ @StefanoErmon @Xiaojie_Qiu 🧵⤵️

Terrific work from @probablynotaz9 !

@elon_lit is indeed GOATed

1/ 🚨 New paper! DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking We give masked diffusion models (MDMs) proper likelihood — and therefore proper perplexity — for the first time. Turns out MDMs are closer to autoregressive models than previously thought.

@siddancha @StefanoErmon @elon_lit @Jose_Blanchet @thanawatsornwan @lutong_hao Thanks for sharing! Will check out

Really cool work @therealgabeguo! 👏👏 Looking forward to reading all the fun details in the paper! We did something similar for robot action sequences in continuous time and space: x.com/siddancha/stat… (streaming-flow-policy.github.io) that unifies autoregressive and diffusion-based generation, but using ODEs instead of SDEs, and has some of the advantages you stated.

🚨New time series generative model just dropped. Paper: arxiv.org/abs/2604.27443 Demo: abc-diffusion.github.io ⏰Meet ABC: Any-Subset Autoregressive Diffusion Bridges in Continuous Time & Space. With @StefanoErmon @elon_lit @Jose_Blanchet @thanawatsornwan @lutong_hao

🔥Empirically, ABC generates high quality videos, weather forecasts, & more. We look forward to unlocking its potential for scientific applications at scale. 🙏Thanks to @ENERGY for funding this research, and @StanfordHAI, @nvidia, & @NERSC for generous compute donations.🇺🇸🌲

🌟Furthermore, ABC is a unification of the two dominant generative modeling paradigms: autoregressive and diffusion models. It extends autoregressive modeling to continuous time and space, and extends diffusion models to the non-Markovian case.

We developed a unified theory of generalization in deep learning. It explains grokking, double descent, benign overfitting, and implicit bias. But theory is only half the story. It turns out that optimizing the population risk of any neural network amounts to a small change to your optimizer. 🧵

GOAT has been accepted to ICML! See you in Seoul🔥🐐🐐

@xavierjgonzalez Great work from my esteemed friend!

Parallelizing nonlinear RNNs is gaining traction! More efficient than transformers; more expressive than linear RNNs. My PhD thesis provides an intro guide to the math (Newton's method) behind the parallelization. Great as a quick-start if you want to explore this new field!

Happy 🦃 Thanksgiving weekend! 🍂 This year, we cooked up a new recipe for juicy fact-storing MLPs. Instead of picking apart trained models, we asked: Can we construct fact-storing MLPs from scratch? 🤔 Spoiler: we can & we figured out how to slot these hand-crafted MLPs into Transformer blocks as modular fact stores! 🧩 New work with @garctrob @ronnygjunkins @jerrywliu @dylan_zinsley @EyubogluSabri Atri Rudra @HazyResearch! 🧵👇