Max Bartolo

@nathanbenaich @robertarail @raais @GoogleDeepMind Roberta is awesome! 🤩

We're excited to have @robertarail at @raais 2026! Roberta leads open-ended learning at @GoogleDeepMind - from MLGym and MaestroMotif to Toolformer and Llama 3's tool use, her work is shaping what it takes for ai agents to keep acquiring skills. Join us on 12 june in london!

Londonmaxxing 🇬🇧

Gemini 3.1 Flash-Lite is available now! It takes an unbelievable amount of complex engineering to make AI feel instantaneous, enabling exciting new frontiers for experimentation!

@jeankaddour Data order maybe where baseline is random shuffle and non-baseline is some form of curriculum?

ML interview question: What is happening here?

Today we’re releasing a preview of Gemini 3.1 Pro and making it available to our users and developers. Very excited to bring the upgraded core we used in Deep Think to everyone. Learn more about Gemini 3.1 Pro: blog.google/innovation-and…

@KanishkaNarayan The tax system that makes it too expensive for tech employees to exercise their own hard-earned equity, handing millions back to foreign companies? That tax system?

“𝘛𝘩𝘦 𝘜𝘒 𝘩𝘢𝘴 𝘵𝘩𝘦 𝘵𝘢𝘭𝘦𝘯𝘵, 𝘵𝘩𝘦 𝘵𝘢𝘹 𝘴𝘺𝘴𝘵𝘦𝘮, 𝘵𝘩𝘦 𝘷𝘪𝘴𝘢𝘴, 𝘵𝘩𝘦 𝘰𝘱𝘵𝘪𝘰𝘯 𝘴𝘤𝘩𝘦𝘮𝘦𝘴, 𝘌𝘐𝘚 𝘢𝘯𝘥 𝘚𝘌𝘐𝘚, 𝘢𝘭𝘭 𝘵𝘩𝘦𝘴𝘦 𝘨𝘳𝘦𝘢𝘵 𝘨𝘰𝘷𝘦𝘳𝘯𝘮𝘦𝘯𝘵 𝘪𝘯𝘵𝘦𝘳𝘷𝘦𝘯𝘵𝘪𝘰𝘯𝘴 𝘵𝘩𝘢𝘵 𝘮𝘢𝘬𝘦 𝘉𝘳𝘪𝘵𝘢𝘪𝘯 𝘰𝘯𝘦 𝘰𝘧 𝘵𝘩𝘦 𝘣𝘦𝘴𝘵 𝘱𝘭𝘢𝘤𝘦𝘴 𝘪𝘯 𝘵𝘩𝘦 𝘸𝘰𝘳𝘭𝘥 𝘵𝘰 𝘴𝘵𝘢𝘳𝘵 𝘢 𝘤𝘰𝘮𝘱𝘢𝘯𝘺”. Full of energy from time with @t_blom, our AI Ambassador, and exceptional UK AI founders - we are building something special here 🇬🇧 🚀

Yet another massively insightful research contribution by @LisaAlazraki 🤩

Future intelligence will be tailored to individuals not average human feedback. It will be personal, adapt continuously online, and shaped by interaction and experience. Excited to see Sara and Sudip build!

@nickfrosst @CoreWeave @cohere if the typo was intended, it's brilliant. if not, it's subliminally brilliant

@CoreWeave cut their support ticket resolution time in half using @cohere (and had a customer satisfactions core of 4.9) You can too. read how here: cohere.com/customer-stori…

We're looking for Research Scientists and Engineers to join the AI team at @iconicgamesio in London. n London. We have diverse positions including: Model Optimization/Efficient Inference, Open-Endedness/Reinforcement Learning, and Generative Vision & Multimodal Foundational Models. Apply here: ats.rippling.com/en-GB/iconic/j… Learn more about Iconic: iconicgames.io We're a small, growing team with high ownership, crafting the minds that inhabit and shape new worlds for interactive entertainment. If you're into games, blueprinting consciousness, or simply building personas that transmit and evoke feelings and emotions beyond what any chatbot assistant could ever do, this is your place. These openings are also suitable for final-year PhD students with experience in the relevant field. Any questions, my DMs are open.

⚡️⚡️⚡️

Pass by poster 1909 in Hall E if you're interested in learning more about RL for Reverse Engineering Human Preferences from @LisaAlazraki @yichern_tan @jaa_campos @maximilianmozes @MarekRei #NeurIPS2025

Cohere Labs x NeurIPS 2025: “Reverse Engineering Human Preferences with Reinforcement Learning" This work shows that RL-tuned preambles can stealthily boost LLM-as-a-judge scores, revealing a subtle but powerful evaluation vulnerability. Congrats to authors @LisaAlazraki, @yichern_tan, @jaa_campos, @maximilianmozes, @MarekRei, @max_nlp 📜 arxiv.org/pdf/2505.15795

Just landed in sunny San Diego ✈️🇺🇲🌞 for #NeurIPS2025. Ping me if you'd like to chat and drop by our spotlight poster tomorrow (neurips.cc/virtual/2025/l…) if you want to see evidence of LLMs using unintelligible preambles and more exciting findings 🤩

A brief 72hrs in beautiful Montreal 🇨🇦, but absolutely worth it. Energising discussions and the chance to debate ideas with people whose work has shaped the field. Thanks for the invite and for hosting a standout workshop @sivareddyg @Yoshua_Bengio @AaronCourville 🙏

So cool to see @sivareddyg's work exploring where in post-training value alignment occurs using PRISM for evaluation! @hannahrosekirk @DynabenchAI @Prolific @IVADO_Qc @Mila_Quebec

Max Bartolo (@max_nlp) on Robust Real World Agents Many LLMs are great at static benchmarks but in reality they perform worse on real world inputs -- we need benchmarks that beat the AI, so we need adversarial ways to collect benchmarks -- such data will have a richer distribution of complex reasoning required compared to SQuAD style QA takes. -- DynaBench, HLE etc are some examples -- but it is more expensive -- Also applies to agentic LLMs -- usually collecting such data requires human feedback But human feedback is not gold standard. -- people tend to prefer generations that are formatted nicely rather than factual correctness -- for the same model, you can play with assertiveness through prompt. Perceived error rates by humans is higher for models when they are unassertive, but in reality, assertive responses contain more factual errors. Verifiable reward is powerful, to solve some of the problems with human feedback (but reward models trained on human feedback may also resurface these problems) Models fail even after closing the loop due to other reasons. Super interesting example on how many R's in blueberry. We fixed R's in strawberry but now blueberry has a problem. Other parts of the talk: is more tokens/context better, what kind of pretraining data helps reasoning.

Gemini 3 is here with impressive agentic, tool use and function calling capabilities. Can't wait to see what people build with it! 🚀

Just arrived at #EMNLP2025 in Suzhou. Looking forward to meeting with everyone! Will be giving an oral presentation of our paper No Need For Explanations: LLMs Can Implicitly Learn from Mistakes In Context this Friday 7th November at 11.30 am in Hall A108 🎤

Cohere Labs x EMNLP 2025: "No Need for Explanations: LLMs can implicitly learn from mistakes in-context" This work demonstrates that eliminating explicit corrective rationales from incorrect answers improves Large Language Models' performance in math reasoning tasks, outperforming chain-of-thought prompting and suggesting that over-constraining models with detailed explanations limits their ability to learn effectively from mistakes. Congrats to authors @LisaAlazraki, @maximilianmozes, @jaa_campos , @yichern_tan, @MarekRei, @max_nlp 📜 arxiv.org/abs/2502.08550