Luca Zappella
57 posts
None of this would have been possible without an extraordinary team from . Huge shoutout to everyone that pushed extrememly hard to make this happen: @LouisBAlgue, @victorturrisi, @kayembruno, @prlz77, @LokeshBoomi, Nikhil Bhendawade, @AmitisShidani1, @JorisPelemans, @theo_olausson, Devon Hjelm, Paul Dixon, @joaomonteirof, @PierreAblin, Vishnu Banna, @ArnoBlaas, Nick Henderson, Kari Noriy, @danbusbridge, @jmsusskind, Marco Cuturi, Irina Belousova, @luca_zapp, and Russ Webb. Particular shout out to @LouisBAlgue and @victorturrisi for the many sleepless nights working together! This was a genuine team effort - the kind of collaborative, careful science I feel fortunate to be part of. Paper: arxiv.org/abs/2602.21472
We propose new scaling laws that predict the optimal data mixture, for pretraining LLMs, native multimodal models and large vision encoders ! Only running small-scale experiments is needed, and we can then extrapolate to large-scale ones. These laws allow 1/n 🧵
𝗣𝗮𝗿𝗮𝗥𝗡𝗡: 𝗨𝗻𝗹𝗼𝗰𝗸𝗶𝗻𝗴 𝗣𝗮𝗿𝗮𝗹𝗹𝗲𝗹 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗼𝗳 𝗡𝗼𝗻𝗹𝗶𝗻𝗲𝗮𝗿 𝗥𝗡𝗡𝘀 𝗳𝗼𝗿 𝗟𝗟𝗠𝘀 For years, we’ve given RNNs for doomed, and looked at Transformer as 𝘁𝗵𝗲 LLM—but we just needed better math 📄arxiv.org/abs/2510.21450 💻github.com/apple/ml-parar…
🚀 Excited to share LinEAS, our new activation steering method accepted at NeurIPS 2025! It approximates optimal transport maps e2e to precisely guide 🧭 activations achieving finer control 🎚️ with ✨ less than 32 ✨ prompts! 💻github.com/apple/ml-lineas 📄arxiv.org/abs/2503.10679
Is the mystery behind the performance of Mamba🐍 keeping you awake at night? We got you covered! Our ICML2025 paper demystifies input selectivity in Mamba from the lens of approximation power, long-term memory, and associative recall capacity. arxiv.org/abs/2506.11891
Our work on fine-grained control of LLMs and diffusion models via Activation Transport will be presented @iclr_conf as spotlight✨Check out our new blog post machinelearning.apple.com/research/trans…
I am so excited to be attending my first @NeurIPSConf this year!! Hit me up if you would like to chat. On Sunday, I will be at the MINT workshop on model interventions. Join us to understand the inner workings of foundation models. sites.google.com/view/mint-2024
Thrilled to share the latest work from our team at @Apple where we achieve interpretable and fine-grained control of LLMs and Diffusion models via Activation Transport 🔥 📄 arxiv.org/abs/2410.23054 🛠️ github.com/apple/ml-act 1/9 🧵
I’m thrilled to announce 3 #internship openings @Apple ML Research in beautiful ☀️ #Barcelona ☀️ for 2025! Two internships on Generative Models (GM), Controllability, Interpretability, and Model Editing; and one on GM &🔈Spatial Audio. Apply: jobs.apple.com/en-us/details/… Details 🧵
