Emily Cheng @ ICLR

DEADLINE APPROACHING! Apply to Brains, Minds, and Machines by March 23. This three-week course is designed for graduate students, postdocs, and faculty in computer science or neuroscience. 🔗More info here: go.mbl.edu/BMM

@zxlzr Really cool work! We found similar results in our ICLR 2026 paper, where more granular control goals hurt both controllability and the sample complexity needed to estimate controllable sets. Might be relevant: openreview.net/forum?id=HJTFg…

How controllable is a Large Language Model, really? 🧐 We often prompt LLMs to "be polite" or "act like a pirate," but the gap between intent and instantiation remains a black box. Introducing our latest work SteerEval: “How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities” 📄 Paper: huggingface.co/papers/2603.02… 💻 Code: github.com/zjunlp/EasyEdi… 📊 Datasets: huggingface.co/datasets/zjunl… 🛠️ What is SteerEval? It’s not just a dataset—it’s a domain-extensible, hierarchical conceptual framework for automatic benchmark synthesis. By combining this automated pipeline with rigorous human verification, we introduce a principled benchmark to audit LLM controllability across 4 domins, including Language Features (Form), Reasoning Patterns (Thought), Sentiment (Emotion), and Personality (Soul). 📊✨ 🧠 Grounded in Marr’s Three-Level Theory To bridge the "intent-realization gap," we borrow from arr’s three levels of analysis, reframing LLM behavior into a Triple-Level Specification (L1-L3): 🎯 L1: Computational Level (What to express) The behavioral goal/intent (e.g., "Be Enthusiasm"). ⚙️ L2: Algorithmic Level (How to express it) The behavioral strategy & patterns (e.g., "Use active voice and energized praise"). ✍️ L3: Implementational Level (How to instantiate it) The physical textual realization (e.g., "Must includes 'hooray' twice"). 🔍 Key Findings: The "Granularity Gap" 📉 Our evaluation of many steering methods reveals a striking "Granularity Gap": Steered LLMs may follow high-level commands (L1) while failing to maintain the underlying behavioral DNA at the implementational level (L3). Surface-level obedience ≠ Deep-level control. 💡 🚀 Why it matters? Structured Auditing: Provides a "mechanistic map" for behavioral safety. 🗺️ Scalable Synthesis: The framework allows researchers to easily extend SteerEval to new behavioral domains. 🏗️ Beyond Prompting: Shifts the focus from "black-box prompting" to "fine-grained behavioral engineering." 🧬 We hope SteerEval serves as a foundation for building LLMs that are not just powerful, but truly predictable and faithful to human intent. 🤝 Would you like to see how your model performs on the L1-L3 hierarchy? Let’s chat! 💬 #Steering #SteerEval #KnowledgeEditing #AI #NLP #LLMs

Autoregressive models dominate, but what if we treat multimodal generation as discrete order agnostic iterative refinement? Excited to share our systematic study on the design space of Tri-Modal Masked Diffusion Models (MDMs). We pre-trained the first Tri-Modal MDM from scratch on (text,), (image, text), and (audio, text). The same model can do ASR, TTS, T2I, captioning and native text generation. What I'm the most proud of in this work is the scientific rigor. Over 3,500 training runs. Principled hyperparameter transfer. Honest results. Carefully controlled ablations across multiple different axis of entanglement. A thread on our empirical findings (arXiV: arxiv.org/abs/2602.21472)

Our Stretch-and-Squeeze (SnS) paper has been accepted to #ICLR2026 @iclr_conf ! Credits to all the other authors for their great work: @LTausani, P. Muratore, @MorganBDTalbot, @giacomo_amerio and @gkreiman . Here’s a thread explaining SnS (full paper: arxiv.org/abs/2506.17040) 1/5

We’re so glad to be sharing this new project today at @NeuripsConf: “Scaling and Context Steer LLMs along the Same Computational Path as the Human Brain.” 📍 Come chat at Hall C–E, Poster #2006, from 4 PM 💫🧠

I'll be at #NeurIPS2025 presenting w/: 1️⃣ Eshani Patel and @yisongyue on generalizing models to input preprocessing 2️⃣ Chris Sandino et al., on new temporal reps of EEG 3️⃣ @shivashriganesh et al., on new iEEG training frameworks 📍Brain & Body FM on Sat 😍 🧠? Let's chat!! 🔗⬇️

If you liked our calibration paper and want to work with me & our team, please apply to this PhD internship. 6-months in our Paris office:

@weiyinko_ml @singhshiviii @beyzaermis Cohere Labs Connect Conference Lightning Talk ⚡️ How Good Are LLMs at Multi-Session Coding Interactions? with @carraznathanael @mhamdy_res They'll explore how well LLMs maintain continuity and context across multiple coding sessions. 📜arxiv.org/abs/2502.13791

𝗣𝗮𝗿𝗮𝗥𝗡𝗡: 𝗨𝗻𝗹𝗼𝗰𝗸𝗶𝗻𝗴 𝗣𝗮𝗿𝗮𝗹𝗹𝗲𝗹 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗼𝗳 𝗡𝗼𝗻𝗹𝗶𝗻𝗲𝗮𝗿 𝗥𝗡𝗡𝘀 𝗳𝗼𝗿 𝗟𝗟𝗠𝘀 For years, we’ve given RNNs for doomed, and looked at Transformer as 𝘁𝗵𝗲 LLM—but we just needed better math 📄arxiv.org/abs/2510.21450 💻github.com/apple/ml-parar…

🚀 Excited to share LinEAS, our new activation steering method accepted at NeurIPS 2025! It approximates optimal transport maps e2e to precisely guide 🧭 activations achieving finer control 🎚️ with ✨ less than 32 ✨ prompts! 💻github.com/apple/ml-lineas 📄arxiv.org/abs/2503.10679

I am hiring a PhD student to start my lab at @ucl! Get in touch if you have any questions, the deadline to apply through ELLIS is 31 October. More details🧵

New paper from my Meta internship! 🚀 We explored dual-architecture communication for latent reasoning in LLMs ☯️ —accepted at the #NeurIPS2025 Foundations of Reasoning in LLMs workshop. Paper: arxiv.org/pdf/2510.00494 1/9 🧵

Induction heads are surprisingly powerful. In a new preprint, we find that they can learn what to attend to in-context! We study this in a hierarchical prediction task and uncover a possible mechanism giving rise to in-context learning in induction heads. See thread for details!

LLMs as a synthesis between symbolic and distributed approaches to language 💯 arxiv.org/abs/2502.11856

The Center for Brains, Minds & Machines [2013-2025]

In our new paper, we explore how we can build encoding models that are both powerful and understandable. Our model uses an LLM to answer 35 questions about a sentence's content. The answers linearly contribute to our prediction of how the brain will respond to that sentence. 1/6

Our "Competitions of Mechanisms" paper proposes an interesting way to interpret LLM behaviors thru how it handles multiple conflicting mechanisms. E.G., in-context knowledge vs. in-weights knowledge🧐This is an elegant philophical way of thinking --

I will be a SPAR mentor this Fall🤖 Check out the programme and apply by 20 August to work with me on formalising and/or measuring and/or intervening on goal-directed behaviour in AI agents More info on potential projects here 🧵

Uncertainty quantification (UQ) is key for safe, reliable LLMs... but are we evaluating it correctly? 🚨 Our ACL2025 paper finds a hidden flaw: if both UQ methods and correctness metrics are biased by the same factor (e.g., response length), evaluations get systematically skewed

A personal update. bhaskar-mitra.github.io/posts/2025/07/…