Zhe Wang

(1/6) Can Autoregressive Models (ARMs) go beyond fixed or random generation orders and learn the optimal order to generate new samples? We believe so! We are excited to present our paper, “Learning-Order Autoregressive Models with Application to Molecular Graph Generation,” at ICML 2025 in Vancouver on July 18th. Feel free to drop by! In our new paper, we introduce Learning-Order Autoregressive Models (LO-ARMs), which are trained to find a consistent, context-dependent generation order on their own. Our model matches and surpasses state-of-the-art results on two major molecular graph generation tasks. Read our paper here: arxiv.org/abs/2503.05979 Joint work with @thjashin, Nicolas Heess, @ArthurGretton, Michalis K. Titsias More in 🧵👇

today, we’re open sourcing the largest egocentric dataset in history. - 10,000 hours - 2,153 factory workers - 1,080,000,000 frames the era of data scaling in robotics is here. (thread)

📢 Job alert We are looking for a Postdoctoral Fellow to work with @ArthurGretton on creating statistically efficient causal and interaction models with the aim of elucidating cellular interactions. ⏰Deadline 27-Aug-2025 ℹ️ ucl.ac.uk/work-at-ucl/se…

Nice writeup on CTMC and discrete diffusions

Google DeepMind has been leading the revolution in Foundation Models for Robotics. Come join us. We are hiring!

Accelerated Diffusion Models via Speculative Sampling, at #icml25 ! at 16:30 Tuesday July 15 poster E-3012 arxiv.org/abs/2501.05370 @ValentinDeBort1 @agalashov @ArnaudDoucet1

Distributional diffusion models with scoring rules at #icml25 Fewer, larger denoising steps using distributional losses! Wednesday 11am poster E-1910 arxiv.org/pdf/2502.02483 @agalashov @ValentinDeBort1 Guntupalli @zhouguangyao @sirbayes @ArnaudDoucet1

@YuchenZhu_ZYC Thanks! Shall we catch up in person if you're also at ICML? Feel free to let me know if you have any questions.

@zhexwang Amazing work!

(1/6) Can Autoregressive Models (ARMs) go beyond fixed or random generation orders and learn the optimal order to generate new samples? We believe so! We are excited to present our paper, “Learning-Order Autoregressive Models with Application to Molecular Graph Generation,” at ICML 2025 in Vancouver on July 18th. Feel free to drop by! In our new paper, we introduce Learning-Order Autoregressive Models (LO-ARMs), which are trained to find a consistent, context-dependent generation order on their own. Our model matches and surpasses state-of-the-art results on two major molecular graph generation tasks. Read our paper here: arxiv.org/abs/2503.05979 Joint work with @thjashin, Nicolas Heess, @ArthurGretton, Michalis K. Titsias More in 🧵👇

@GatsbyUCL @KevinHanHuang1 Congratulations!! @KevinHanHuang1

🎊 Congratulations to Kevin Huang (@KevinHanHuang1) on passing his PhD viva with minor corrections! 🥳 🔖 Universality beyond the classical asymptotic regime

(6/6) Finally, can LO-ARMs generate "better" molecules? Yes, they can. On both the QM9 and ZINC250k datasets, LO-ARMs match or exceed state-of-the-art (SOTA) results. Performance was evaluated across key metrics for distribution similarity and drug-likeness.

(5/6) Can LO-ARMs learn consistent and meaningful generation orders on their own? Yes — and the strategy/generation order they discover is surprisingly intuitive. On two major molecular generation tasks (QM9 and ZINC250k), our models learned a consistent, two-phase process: 1. Build the Skeleton: First, generate the chemical bonds. 2. Infill the Atoms: Then, add the atoms to the skeleton. We find that 99% of the new samples follow this generation order. Here’s an example of this in action for generating a ZINC250k molecule. We start with a blank slate where all tokens are masked, and in each step, the model adds one new piece — either an atom or a bond, until the molecule is complete.

@ZoubinGhahrama1 @geoffreyhinton Just took the Gatsby courses last year (now they are split to two modules approximate inference and unsupervised learning), and the courses are incredibly helpful. I even read your matlab code!😁

This is the syllabus of the course @geoffreyhinton and I taught in 1998 at the Gatsby Unit (just after it was founded). Notice anything?

📢 We have an opportunity for students to join our PhD programme in Theoretical Neuroscience and Machine Learning this September. Application deadline is 27 May 2025. Information & how to apply 👉 ucl.ac.uk/gatsby/study-a…

We are hiring a student researcher at Google DeepMind to work on fundamental problems in discrete generative modeling! Examples of our recent work: masked diffusion: arxiv.org/abs/2406.04329 learning-order AR: arxiv.org/abs/2503.05979 If you find this interesting, please send an email to: {jiaxins,mtitsias} AT google DOT com

@karl_tuyls Congratulations Karl!

I’m happy to share that I’m starting a new position as Director of AI Research at Meta! Looking forward to advancing the next generation of Autonomous AI Agents with the Llama team. Bringing 25 years of experience in AAMAS to develop impactful open-source action and agent models. Stay tuned!

Excited to share details of AlphaGeometry2 (AG2), part of the system that achieved silver-medal standard at IMO 2024 last July! AG2 now has surpassed the average gold-medalist in solving Olympiad geometry problems, achieving a solving rate of 84% for all IMO geometry problems over the last 25 years, compared to 54% previously! Figure is a crazy problem solved elegantly by AG2 (for the first time according to our knowledge!). Will follow up with more details later in a thread. Paper: arxiv.org/abs/2502.03544 Nature article: nature.com/articles/d4158… Authors: Yuri Chervonyi, Trieu H. Trinh, Miroslav Olsak, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V. Le, Thang Luong Special thanks to Dawsen Hwang, Edward Lockhart, and Steven Creech for contributions to the development of AlphaGeometry2. We would also like to thank Yifeng Lu, Henryk Michalewski, Ed Chi, David Silver, Pushmeet Kohli, and Demis Hassabis for their thoughtful discussions, help and support.

Still a week to submit your work to ICLR 2025 workshop on world models!