Ke Wang

Cool news: our extended Riemannian Gaussian VFM paper is out! 🔮 We define and study a variational objective for probability flows 🌀 on manifolds with closed-form geodesics. @FEijkelboom @a_ppln @CongLiu202212 @wellingmax @jwvdm @erikjbekkers 🔥 📜 arxiv.org/abs/2502.12981

Bit late for the announcements but very happy to share that MEMOIR is accepted to Neurips 2025🎉! Great collaboration with @qinym710 @nikdimitriadis, @alesfav, @pafrossard! See you in San diego!

🚨 Incredibly excited to share that I'm starting my research group focusing on AI safety and alignment at the ELLIS Institute Tübingen and Max Planck Institute for Intelligent Systems in September 2025! 🚨 Hiring. I'm looking for multiple PhD students: both those able to start in Fall 2025 (i.e., as soon as possible) and through centralized programs like CLS, IMPRS, and ELLIS (the deadlines are in November) to start in Spring–Fall 2026. I'm also searching for postdocs, master's thesis students, and research interns. Fill the Google form below if you're interested! Research group. We will focus on developing algorithmic solutions to reduce harms from advanced general-purpose AI models. We're particularly interested in alignment of autonomous LLM agents, which are becoming increasingly capable and pose a variety of emerging risks. We're also interested in rigorous AI evaluations and informing the public about the risks and capabilities of frontier AI models. Additionally, we aim to advance our understanding of how AI models generalize, which is crucial for ensuring their steerability and reducing associated risks. For more information about research topics relevant to our group, please check the following documents: - International AI Safety Report, - An Approach to Technical AGI Safety and Security by DeepMind, - Open Philanthropy’s 2025 RFP for Technical AI Safety Research. Research style. We are not necessarily interested in getting X papers accepted at NeurIPS/ICML/ICLR. We are interested in making an impact: this can be papers (and NeurIPS/ICML/ICLR are great venues), but also open-source repositories, benchmarks, blog posts, even social media posts—literally anything that can be genuinely useful for other researchers and the general public. Broader vision. Current machine learning methods are fundamentally different from what they used to be pre-2022. The Bitter Lesson summarized and predicted this shift very well back in 2019: "general methods that leverage computation are ultimately the most effective". Taking this into account, we are only interested in studying methods that are general and scale with intelligence and compute. Everything that helps to advance their safety and alignment with societal values is relevant to us. We believe getting this—some may call it "AGI"—right is one of the most important challenges of our time. Join us on this journey!

🚀 Big time! We can finally do LLM RL fine-tuning with rewards and leverage offline/off-policy data! ❌ You want rewards, but GRPO only works online? ❌ You want offline, but DPO is limited to preferences? ✅ QRPO can do both! 🧵Here's how we do it:

Excited to present #DeFoG with @qinym710 at #ICML2025 ! Catch our oral today at 3:30 PM (West Exhibition Hall C) and join us at the poster after (4:30–7:00 PM, East Exhibition Hall A-B #E-3004). Come chat graphs & generative models!

🚀 Presenting #DeFoG: our discrete flow‑matching framework for graph generation! Catch our #ICML2025 oral presentation today (3:30 – 3:45 PM, in West Exhibition Hall C) and drop by the poster right after (4:30 –7:00). Come chat graphs & generative models! @manuelmlmadeira

🧬 New roadmap out in Nature Reviews Molecular Cell Biology! 🤖 We show how RNA-LMs + GNNs can come together to model the RNA interactome & uncover new roles for non-coding RNA. 💊 Clinical links to RNA therapies for cancer & neuro diseases. 📄 Read it: bit.ly/4kNblk6

@_simonsmith In contrast, our method updates the model weights directly to inject knowledge, enabling it to generalize across semantically related prompts, as demonstrated by our SOTA performance on the generalization metric.

This looks interesting, but how does it compare to simply having facts stored in a database for retrieval? Like, why store updated facts in the model at all?

@_simonsmith For instance, it can fail to recognize rephrased edits, such as from "Where is the Eiffel Tower?" to "Which city is the Eiffel Tower located in?"

@_simonsmith While GRACE achieves near-perfect accuracy, its reliance on explicit storage limits its ability to generalize to rephrased prompts regarding the same edited knowledge.

@_simonsmith Thanks for liking and sharing our work! One of the baselines in our comparisons, GRACE, closely resembles what you described. It stores edited facts by directly saving modified output patterns in an external dictionary.

@MichelIvan92347 Thank you for liking and sharing our work!

MEMOIR = interesting work on lifelong model editing here imo from an EPFL team 👇 Bravo to the team ! 👏

@kaelrickwyn Thank you for liking and sharing our work!

@Machine_Enginee @qinym710 @nikdimitriadis @alesfav @pafrossard plt.rcParams.update({ "text.usetex": True, "font.family": "serif", "font.serif": ["cm10"], "font.size": 14, })

@qinym710 @wangkeml @nikdimitriadis @alesfav @pafrossard The plots are beautiful and sharp, what plotting library did you use? I sense some kind of LaTeX tweak. Can I get the code for it? Thanks

This lets MEMOIR generalize to rephrased prompts. It rewrites facts like "The Eiffel Tower is in Berlin" and still answers correctly when asked "What is the location of the Eiffel Tower?" 🗼✅ 🧵 4/7

@Machine_Enginee @qinym710 @nikdimitriadis @alesfav @pafrossard Thank you for your kind words about our plots! We created them directly using Matplotlib, configuring the fonts to match those commonly used in LaTeX. Below are the font settings we applied in Matplotlib:

Very happy to share our new work: MEMOIR 📖! MEMOIR is a lifelong model editing method to edit factual knowledge in LLMs with minimal forgetting even with thousands of edits!

How can we inject new knowledge into LLMs without full retraining, forgetting, or breaking past edits? We introduce MEMOIR 📖— a scalable framework for lifelong model editing that reliably rewrites thousands of facts sequentially using a residual memory module. 🔥 🧵1/7

This was an amazing collaboration with @wangkeml, @nikdimitriadis, @alesfav, @pafrossard! 📜 Paper: arxiv.org/abs/2506.07899 Looking forward to seeing what MEMOIR helps you rewrite next! 🧵 7/7

Happy to share that DeFoG: Discrete Flow Matching for Graph Generation will be showcased as a Spotlight Poster at #ICML2025 ! -> Explore the paper: arxiv.org/abs/2410.04263 -> Open-source code: github.com/manuelmlmadeir… Looking forward to your feedback on our repository!

Happy to release the code for DeFoG (github.com/manuelmlmadeir…), accepted as an ICML 2025 Spotlight Poster, with @manuelmlmadeira! → Efficient training & sampling → Wide dataset support & extensible dataloader → Hydra-powered CLI & WandB → Docker/Conda setup