Luca Eyring

Reward hacking is challenging when fine-tuning few-step Diffusion models. Direct fine-tuning on rewards can create artifacts that game metrics while degrading visual quality. We propose Noise Hypernetworks as a theoretically grounded solution, inspired by test-time optimization.

What if discrete diffusion didn’t have to be stuck with mask or uniform noise? 🤔 In our new paper, we show how to go beyond them, unlocking much richer noising processes. And the empirical results are surprisingly strong! 🚀 🌐 Project Page: oussamazekri.fr/gdds 📑 Paper: arxiv.org/pdf/2603.21342 💻 Code: github.com/ozekri/gdds Thread below 🧵

Thanks for the acknowledgement and cool work on VFMs! My view of VFM vs HyperNoise would be that in practice VFM can probly achieve stronger and more efficient reward alignment due to additional fine-tuning of the generator. However, its limitation is that it requires real data to train while HyperNoise operates completely data free. Also, fine-tuning the generator could make VFM more prone to reward hacking. For inverse problems, the goal is very strong reward alignment while reward hacking is not rlly a problem. So I fully agree with VFM being better aligned with this setting! For general preference alignment, e.g. in T2I the data requirement might hinder VFMs applicability in practice, although I would be quite curious to see how VFM and HyperNoise compare in that setting, and whether reward hacking occurs. Happy to discuss more about this!

Thanks for pointing both of these out, @yuanzhi_zhu! We will definitely cite Noise Hypernetworks and Golden Noise in our next revision, as both are highly relevant to our core premise. To give a bit of context on how VFM differs from Noise Hypernetworks: we extend our method to inverse problems where the input distribution changes, meaning we can't easily rely on a pre-trained model to act as the adapter. Also, because we use a tractable Gaussian variational family, we don't require assumptions on the Lipschitz constant. To compensate for the limited expressivity of the Gaussian, we train both the adapter and the flow map jointly. Thanks again for flagging these!

🔊 Make Some Noise! Introducing Variational Flow Maps (VFM) - a principled framework for one-step conditional image generation & reward alignment. No iterative guidance. No hundreds of NFEs. Just learn the right noise 🎯 🚀 Below are one-step generation results from VFM after only 0.5 epoch of reward fine-tuning on ImageNet ⚡ Paper: arxiv.org/abs/2603.07276 Code: github.com/abbasmammadov/… 🧵1/10

🚨 New paper! arxiv.org/abs/2603.06610 Are you sure your post-trained LLM isn’t forgetting something? Adapting LLMs is known to cause forgetting. We usually measure it via general knowledge benchmarks. But if MMLU doesn’t drop…. are you really fine?

Excited to show some surprising inventions on generative multiplayer games we made at Google with Stanford. We call the work MultiGen. I've always been inspired by early studios like id Software with Doom or Blizzard with Warcraft bringing networked video games to the next level. We are at the point in history where we can make strides like them, but for generative games. It's a strange feeling to be in the age of generative video games while still discovering how exactly to train the models and design the tools that make them useful. All of the tools that have been invented for classic game engines need to be redesigned for generative games. For example level and world design is not entirely possible with existing technology. We introduce editable memory to diffusion game engines that allow for design of new levels via a minimap. But we can easily imagine how this can be expanded with different creation tools. The end goal of this research direction is to allow game designers to be able to guide the generation process of their world, at the granularity that they prefer. Editable memory also allows us to add multiplayer to Generative Doom. We were amazed when we saw GameNGen some years ago, and now you can play it live with friends in real-time, on your couch or even online. Shared representations like our editable memory seem like the future for this type of experience. Models are, in some cases, expensive and approximate encoders but great interpolators and extrapolators. Leveraging their strengths lets you have completely new experiences that can be realized now and not in the distant future. This work was started at my previous team and continued in collaboration with Stanford. Congratulations to all for the discoveries.

Very fortunate to have the opportunity to contribute to this project! It's so clean and well structured, and I truly learned a lot while building it along the way! If you are working on diffusion distillation (or related topics), this is **the** codebase to refer to!

Check out our new work on getting more out of your Vision Transformers without fine-tuning by leveraging intermediate representations when probing! We find that attentive probing is most robust for fusing across layers while showing exactly which layers matter for what task 👇

🚨 Looking for a fully self-contained intro to diffusion models that covers both continuous (images) and discrete (text, sequences) data? 🆕 We just released: “Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction” arxiv : arxiv.org/abs/2512.05092 S/o to @andrea_dittadi for his amazing support & guidance, and huge thanks to @TobiasHppe1, @k_neklyudov, @AlexanderTong7 and @stefanAbauer for their supervision! 🙌 One roadmap for all of diffusion. 🏎️💨 After a few failed posts, broken previews, and getting briefly flagged by X… the full thread's finally out 🤯🧵👇

We released a self-contained diffusion intro! We present continuous and discrete side-by-side to highlight the parallel math, and color-coded boxes 🔵🔴🟡 allow you to speed-run the whole thing. Super proud of @vincentpaulinef for an outstanding job!!🎉 arxiv.org/abs/2512.05092

What an excellent self-contained, formal introduction to diffusion by @vincentpaulinef!!! It's fantastic to see write-ups like this, written with the author's interest in understanding and compressing the literature -- for their own enjoyment and interest. To see patterns. Congrats also to my pal @andrea_dittadi arxiv.org/abs/2512.05092

Diffusion Circle happening this afternoon!

Excited to be in San Diego at #NeurIPS2025 this week. We'll be presenting Noise Hypernetworks at Hall C,D,E #3605 Thu 4 Dec 11am! If you're interested in Diffusion/Flow Matching or Reward Alignment, I'd love to chat! Feel free to DM me if you'd like to connect.

@aakaran31 Finally got to read it now, really cool analysis in there! And indeed, I think one might even be able to train like a more general hypernetwork on multiple of these tasks together.

@LucaEyring Super cool work!! I'd be curious to see how these hypernetworks perform when the rewards are measurement errors for e.g. inpainting or deblurring tasks! We had a similar finding that informed noise initializations make a huge difference for these tasks. (arxiv.org/abs/2506.10955)

Reward hacking is challenging when fine-tuning few-step Diffusion models. Direct fine-tuning on rewards can create artifacts that game metrics while degrading visual quality. We propose Noise Hypernetworks as a theoretically grounded solution, inspired by test-time optimization.

⭐️New Paper⭐️ just in time for NeurIPS. Presenting: Test-time scaling of diffusions with flow maps. project: flow-map-trajectory-tilting.github.io arXiv: arxiv.org/abs/2511.22688 Want to properly adapt your diffusion at inference time with a reward? You should do everything at the level of the flow map. With the amazing @amsabour, @cdomingoenrich, @nmboffi, @FidlerSanja , @karsten_kreis, and @evdende2.

We’re releasing UNI-D², a unified codebase for discrete diffusion language models 🤝🚀 Co-led with @vincentpaulinef and an amazing advisor team: @stefanAbauer, @AlexanderTong7 , @andrea_dittadi, @AMK6610, @KaplFer 🙌 🔗 GitHub: github.com/nkalyanv99/UNI… 📚 Docs: nkalyanv99.github.io/UNI-D2/ Reproduce and extend state-of-the-art baselines with one toolkit. Let’s move beyond autoregressive models and push discrete diffusion together 🧵👇

🚀 New paper! arxiv.org/abs/2511.16655 Recently, Cambrian-S released models & two benchmarks (VSR & VSC) for “spatial supersensing” in video! We found: 1️⃣ Simple no-frame baseline (NoSense) ~perfectly solves VSR! 2️⃣ Tiny sanity check collapses Cambrian-S perf to 0% on VSC! 🧵👇

@dvruette @gowthami_s This idea is explored here arxiv.org/abs/2412.03487 in their metric-induced paths and applied at scale here fudoki-hku.github.io. Although it does remain unclear how to get meaningful distances in discrete space (they use distance between normalized token embeddings)

@gowthami_s indeed, that would be interesting! but so far i haven’t seen anybody be able to make this work

most diffusion LLMs out there don’t really do diffusion, they just predict the next token in a randomized order. eventually people will realize that there are much smarter ways to do this.

@g_palla1 @zeynepakata @TU_Muenchen @Googleorg Thanks Giovanni!!:)

@LucaEyring @zeynepakata @TU_Muenchen @Googleorg Congratulations Luca!!!

Honored to receive one of the Google PhD Fellowships 2025 in ML Foundations for my research on "The Role of the Source Distribution in Generative Modeling: Beyond Fixed Gaussian Noise" :) Grateful to @zeynepakata, Alexey, @TU_Muenchen, @Googleorg, and all my collaborators!

@theo_uscidda @zeynepakata @TU_Muenchen @Googleorg Thanks a lot Théo!! :)

@LucaEyring @zeynepakata @TU_Muenchen @Googleorg Congrats Luca!! More than deserved :)