Junyeob Baek

Can AI truly understand the building blocks of visual reality? Our Dreamweaver model breaks down videos into fundamental visual and dynamic elements without auxiliary data, then recombines them to generate novel futures! See it at #ICLR2025 Paper: arxiv.org/abs/2501.14174

A 10 million parameter model just outperformed deterministic rivals 3 times its size by doing something regular recursive AI dont do: exploring multiple reasoning paths at the same time. Most AI reasoning models are trapped on a single train of thought, and GRAM ("Generative Recursive Reasoning") is the first to break that by letting the model think in parallel universes simultaneously. The problem is that all existing recursive models are fully deterministic, meaning given the same input they always follow the exact same reasoning path and can never escape a wrong trajectory or discover more than 1 valid answer. GRAM fixes this by injecting learned randomness at each refinement step, so the model samples a slightly different direction each time rather than snapping to 1 fixed next state, which produces a spread of diverse reasoning trajectories. At test time the model runs many of these paths in parallel and selects the best one using a small reward predictor trained alongside the main model, adding a "width" scaling axis on top of the usual "depth" axis of running more recursion steps. On hard Sudoku puzzles, GRAM with 10M parameters hits 97% accuracy versus 87.4% for the best prior recursive model, and with only 20 parallel samples it outperforms every deterministic baseline even at 320 recursion steps. On tasks with many valid answers like N-Queens, deterministic recursive models collapse as the number of solutions grows, while GRAM maintains near-perfect accuracy throughout. The same stochastic framework also acts as a generator: given a blank board, GRAM produces valid Sudoku puzzles 99% of the time using 16 steps, versus 1,000 steps and 55M parameters for the best diffusion baseline at just 91%. --- Paper Link – arxiv. org/abs/2605.19376v1

A fascinating paper supervised by Yoshua Bengio 👀 "Generative Recursive Reasoning" Test time compute should scale not just by thinking deeper, but by thinking wider. This paper makes recursion generative. It samples many latent reasoning trajectories, letting the model explore multiple hypotheses in parallel, so they don't follow one deterministic path and collapse to one answer. It improves Sudoku, ARC AGI, N Queens, and graph coloring, while also generating valid Sudoku boards and MNIST digits from scratch.

1/ Deterministic AI reasoning has a fatal flaw: once it gets stuck in a local minimum, it can never escape. Traditional recursive models follow a single, fixed latent trajectory. A new paper introduces a way to make reasoning stochastic, unlocking parallel search. 🧵

@AMurthyulas @SungjinAhn_ This paper provides a good analysis of failure cases in latent reasoning, while we show the principled probabilistic method to overcome! They support how a single reasoning path falls into the fake attractor, which leads to failures in reasoning.

@SungjinAhn_ How much different from this work.? x.com/i/status/20120…

🧠We introduce "Generative Recursive Reasoning"! Recursive Reasoning Models like HRM, TRM, and Looped Transformers are deterministic — same input, same reasoning, every time. They collapse the entire space of plausible reasoning paths into a single attractor. Our model GRAM (Generative Recursive reAsoning Models) turns recursion itself into a stochastic latent trajectory. Multiple hypotheses, alternative solution strategies, and inference-time scaling not just by depth, but by width — parallel trajectory sampling. And here's the kicker: the same formulation that gives us conditional reasoning p(y|x) also makes GRAM a general generative model p(x). With only 10M params: • Sudoku-Extreme: 97.0% (TRM 87.4%) • ARC-AGI-1: 52.0% • ARC-AGI-2: 11.1% • N-Queens coverage: 90%+ 📄 Paper: arxiv.org/abs/2605.19376 🌐 Project page: ahn-ml.github.io/gram-website w/ Junyeob Baek @JunyeobB (KAIST), Mingyu Jo @pyross0000 (KAIST), Minsu Kim @minsuuukim (KAIST & Mila), Mengye Ren @mengyer (NYU), Yoshua Bengio @Yoshua_Bengio (Mila), Sungjin Ahn @SungjinAhn_ (KAIST)

Generating Sudoku map. GRAM generates valid maps in less than 10 recursion steps. Diffusion (D3PM) takes much more steps and often leaves incorrect cells.

@enjoyingthewind @SungjinAhn_

@enjoyingthewind @SungjinAhn_ Pls check our ablation study. Only Stochasticity fails at multi-solution tasks and generations.

@enjoyingthewind @SungjinAhn_ Great question! Sharing similar motivation, this seems only adding fixed noise on learned TRM. Our formulation is more general; generative process of recursive reasoning, which learns trajectories distributions via variational approach.

We are seeking a highly motivated postdoctoral researcher to work on fundamental challenges toward AGI, particularly in reasoning, abstraction, and world modeling. The position also offers potential opportunities for co-advising with Yoshua Bengio (Mila) and/or Mengye Ren (NYU). Research areas include: • World Model Learning & Planning • Compositional Generalization & Neuro-Symbolic World Learning • Causal Discovery, Reasoning, and Abstraction This position is supported by the InnoCORE Fellowship Program 2026, with: • Competitive salary of KRW 90M+ (~USD 60K+) • Renewable yearly contract For more information and recent publications: mlml.kaist.ac.kr If you are interested, please send me your CV by email.

Thanks to your incredible support during the closed beta, we have officially launched on the Google Play Store! For those of you who love "Vibe Coding" with Claude, you can now manage your remaining tokens in real-time with our widget. :) App Link: play.google.com/store/apps/det…

“Understanding LoRA as Knowledge Memory” Right now, most research treats LoRA like a cheap fine-tune toggle, but if you want to use it as swappable knowledge memory, the rule of thumb has been mostly vibes. This paper fixes that with a systematic audit of LoRA-as-memory, where it maps how storage scales and saturates with rank. It shows you get way more memory per token by training on QA/summaries instead of raw passages.

Understanding LoRA as Knowledge Memory 🚀 Can we save new LLM facts directly into LoRA weights? While recent works are hastily treating LoRA as a plug-and-play knowledge memory, the fundamental mechanics governing its capacity and composability have remained largely unexplored. 🤯We asked the hard question: Can an adapter meant for task adaptation actually serve as a reliable store for precise, declarative knowledge? To find out, we ran the first systematic empirical study mapping the design space of LoRA-based memory. The shocking reality is that treating LoRA as a memory unit can catastrophically fail in certain settings if you blindly trust it. ✅ Rather than proposing a single architecture, our paper provides practical guidance on its hidden operational boundaries —from characterizing finite storage capacity limits to the harsh realities of multi-module scaling and merging interference. Check out our systematic map of when LoRA memory succeeds, and exactly when it breaks! 🧑🏻‍💻Led by my fantastic students @SeungjuBack (KAIST) and @DongwooLee00 (KAIST), in collaboration with Samsung SDS. arxiv.org/abs/2603.01097

I just found someone don't want to share their email in random tweet. you can also join through group! groups.google.com/g/claudemetert…

Built an app with Claude Code to track Claude usage in real-time. A bunch of people in my lab wanted to try it, so I figured — why not put it on the Play Store? To publish, I need 12 closed testers first. Anyone interested? (Android only!) Drop your email in the comments🙌

@NikolasSapa sure, nikolas. happy to hear your interest. Please drop your email here or through DM to add testers.

@JunyeobB I would love to help

@ravikiran_16 Thank you for your interest and kind words! It would be really helpful if you participated in our closed testing. Please leave your email address here or through DM!

@JunyeobB love this. been wanting something exactly like this to see where the token spend actually goes. happy to test if you need another android user

🧠 Our core question: "How can we extend MCTD to longer, more complex compositional planning tasks, beyond its trained trajectory lengths?" 💡 Our solution (C-MCTD): We solve this problem with plan-level tree search, and boost its efficiency via parallelization and amortization. It has been accepted as a Spotlight at the upcoming #neurips2025 . 📄 ArXiv: arxiv.org/abs/2510.21361 🌐 Project Page: jaesikyoon.com/c-mctd-page/ This work was advised by @SungjinAhn_ and co-worked with a great colleague @hyeonscho . Huge thanks to them and MLML members!

🚨 Check out our new paper on next generation language modeling via "loopholing" discrete diffusion! 🤯 Surprisingly, our loopholing diffusion achieved a huge performance improvement, finally making it match (or even surpass) autoregressive models! ✅ How? We introduce the "loopholing" mechanism — a discrete diffusion that introduces a deterministic bypass alongside the stochastic path to break the sampling wall. 👨🏻‍💻 Led by my fantastic student Mingyu (@pyross0000, KAIST) and @jaesikyoon_ (KAIST), in collaboration with Justin Deschenaux (EPFL) and Caglar Gulcehre (EPFL, Microsoft). 📄 arXiv: arxiv.org/abs/2510.19304 🌐 Project: sites.google.com/view/lddms/home