Alice Bizeul

✨New Preprint ✨ Ever thought that reconstructing masked pixels for image representation learning seems sub-optimal? In our new preprint, we show how masking principal components—rather than raw pixel patches— improves Masked Image Modelling (MIM). Find out more below 🧵

𝗣𝗮𝗿𝗮𝗥𝗡𝗡: 𝗨𝗻𝗹𝗼𝗰𝗸𝗶𝗻𝗴 𝗣𝗮𝗿𝗮𝗹𝗹𝗲𝗹 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗼𝗳 𝗡𝗼𝗻𝗹𝗶𝗻𝗲𝗮𝗿 𝗥𝗡𝗡𝘀 𝗳𝗼𝗿 𝗟𝗟𝗠𝘀 For years, we’ve given RNNs for doomed, and looked at Transformer as 𝘁𝗵𝗲 LLM—but we just needed better math 📄arxiv.org/abs/2510.21450 💻github.com/apple/ml-parar…

🚀 Excited to share LinEAS, our new activation steering method accepted at NeurIPS 2025! It approximates optimal transport maps e2e to precisely guide 🧭 activations achieving finer control 🎚️ with ✨ less than 32 ✨ prompts! 💻github.com/apple/ml-lineas 📄arxiv.org/abs/2503.10679

@syguoML @GatsbyUCL Thanks for sharing the slides, Siyuan! Any plans to come and present your work at ETH?

Understanding intelligence would be today's most interesting challenge. I think physics of learning -- learning follows the laws of physics -- is a start to derive new algorithms. Very much enjoyed the visit @GatsbyUCL and chat about physics of learning. I can feel the buzz for theory! Slides of the talk are attached. docs.google.com/presentation/d…

We’re excited to share our new paper: Continuously-Augmented Discrete Diffusion (CADD) — a simple yet effective way to bridge discrete and continuous diffusion models on discrete data, such as language modeling. [1/n] Paper: arxiv.org/abs/2510.01329

🚨 Machine Learning Research Internship opportunity in Apple MLR! We are looking for a PhD research intern with a strong interest in world modeling, planning or learning video representations for planning and/or reasoning. If interested, apply by sending an email to me at vthilak@apple.com. Applications will be reviewed until the position is filled.

Super excited to share l3m 🚀, a library for training large multimodal models, which we used to build AIM and AIMv2. Massive thanks to @alaa_nouby @DonkeyShot21 Michal Klein @MustafaShukor1 @jmsusskind and many others.

Had a great time speaking at @NECLabsEU about using activation steering for better instruction-following in LLMs! Check out the talk 🗣️: youtu.be/3ozuaGaEjpo?si… and paper 📜: arxiv.org/abs/2410.12877 This work that I did at @MSFTResearch shows how interpretability-based approaches can improve LLM controllability, connecting model understanding with practical utility.

We hosted an insightful talk by @alesstolfo, PhD student at @ETH and doctoral fellow at the Swiss CYD Campus, on improving instruction-following in language models via activation steering. Watch here: youtu.be/3ozuaGaEjpo?si…. #NECLabs #LLM

Join us tomorrow morning #ACL2025NLP #ACL2025

I am heading to @icmlconf to present our position paper with @randall_balestr @klindt_david @wielandbr on what we believe are the important next steps to advance SSL. It's not either theory or practice, it's both. We as a community need a better discussion.

Is the mystery behind the performance of Mamba🐍  keeping you awake at night? We got you covered! Our ICML2025 paper demystifies input selectivity in Mamba from the lens of approximation power, long-term memory, and associative recall capacity. arxiv.org/abs/2506.11891

@ibalazevic @GoogleDeepMind So cool that you were a part of that ! Congrats !

🚀Meet Gemini Diffusion, our first diffusion-based and super fast language model, just announced at Google I/O!🚀 Very excited to be able to share what I've been working on for the past little while with our amazing small team @GoogleDeepMind.

Current KL estimation practices in RLHF can generate high variance and even negative values! We propose a provably better estimator that only takes a few lines of code to implement.🧵👇 w/ @xtimv and Ryan Cotterell code: arxiv.org/pdf/2504.10637 paper: github.com/rycolab/kl-rb

Paper: arxiv.org/abs/2406.18450

Presenting our work at #ICLR this week! Come by the poster or oral session to chat about copyright protection and AI/LLM safety 📌 𝐏𝐨𝐬𝐭𝐞𝐫: Friday, 10 a.m. – 12.30 p.m. | Booth 537 📌 𝐎𝐫𝐚𝐥: Friday, 3.30 – 5 p.m. | Room Peridot @FraPintoML @DonhauserKonst @FannyYangETH

Interested in understanding feature extraction of an amazing self-supervised method, DIET (arxiv.org/abs/2410.21869)? Come to our talk on Sat 11am at Garnet 216-218 and poster #310 at 3pm. My fabulous co-authors: @rpatrik96 @AliceBizeul @randall_balestr @klindt_david @wielandbr.

I am at #ICLR25 😎 and looking for collaborations on self-supervised learning/ understanding representation learning and logical reasoning. Let's chat. 💭 Otherwise, come by on Sat 3pm to our poster #308 on cross-entropy based SSL.

I am at ICLR this week ! Excited to be presenting our work on identifiability with @rpatrik96 @JuhosAttila @randall_balestr @klindt_david @wielandbr. Tune in on Saturday at 11AM in Garnet 216-218 for our oral presentation and at 3PM in Hall 3/2B for our poster!

Made possible by the generous @FlatironInst @SimonsFdn @PolymathicAI support, and our amazing co-organizers @cosmo_shirley @albertobietti and @ylecun !

Check out this post that has information about research from Apple that will be presented at ICLR 2025 in 🇸🇬 this week. I will be at ICLR and will be presenting some of our work (led by @samira_abnar) at SLLM @sparseLLMs workshop. Happy to chat about JEPAs as well!