Anthony Chen

Lots of discourse around long-context language models (LCLMs) subsuming RAG and retrieval but how close are we to this paradigm shift? Introducing LOFT a 1 million token benchmark spanning 6 tasks & 35 datasets to test LCLMs’ ability to do in-context retrieval & reasoning [1/10]

kinda wild that larry page had the bitter lesson figured out in ~2007. for context, sutton published his version in 2019

New paper out today, proving a novel theorem in algebraic geometry with an internal math-specialized version of Gemini. This was a collaboration between @GoogleDeepMind (Professor Freddie Manners and @GSalafatinos, hosted by the Blueshift team) and Professors Jim Bryan, Balazs Elek, and Ravi Vakil. arxiv.org/abs/2601.07222

This is Gemini 3: our most intelligent model that helps you learn, build and plan anything. It comes with state-of-the-art reasoning capabilities, world-leading multimodal understanding, and enables new agentic coding experiences. 🧵

Instructions/reasoning are now everywhere in retrieval - we want embeddings to do it all! 🚀 But... is it even possible? 🤔 Turns out, it's not possible for single-vector models 😱 theoretically and empirically! To make it obvious we OSS a simple eval SoTA models flop on! 🧵

What if you could not only watch a generated video, but explore it too? 🌐 Genie 3 is our groundbreaking world model that creates interactive, playable environments from a single text prompt. From photorealistic landscapes to fantasy realms, the possibilities are endless. 🧵

@yizhongwyz @UTAustin @UTCompSci Congratulations Yizhong!!

Thrilled to announce that I will be joining @UTAustin @UTCompSci as an assistant professor in fall 2026! I will continue working on language models, data challenges, learning paradigms, & AI for innovation. Looking forward to teaming up with new students & colleagues! 🤠🤘

Bon appétit 🍝

@arvind_io @Google @GoogleDeepMind welcome to the team!

thrilled to be back @Google in the @GoogleDeepMind team! The technical breadth and expertise across the whole stack (hardware->infra->deep learning->products) is truly mind-blowing. Great to see a lot of familiar faces and meet new friends. Look forward to learning a lot!

BREAKING: Gemini 2.5 Pro is now #1 on the Arena leaderboard - the largest score jump ever (+40 pts vs Grok-3/GPT-4.5)! 🏆 Tested under codename "nebula"🌌, Gemini 2.5 Pro ranked #1🥇 across ALL categories and UNIQUELY #1 in Math, Creative Writing, Instruction Following, Longer Query, and Multi-Turn! Massive congrats to @GoogleDeepMind for this incredible Arena milestone! 🙌 More highlights in thread👇

Think you know Gemini? 🤔 Think again. Meet Gemini 2.5: our most intelligent model 💡 The first release is Pro Experimental, which is state-of-the-art across many benchmarks - meaning it can handle complex problems and give more accurate responses. Try it now → goo.gle/4c2HKjf

Historic.

GDM's work converting Gemini into a SOTA dual encoder is now out! SOTA results across all benchmarks including exceptionally strong coding performance. Check out the tech report for more details and some interesting ablations!

This is the most gut-wrenching blog I've read, because it's so real and so close to heart. The author is no longer with us. I'm in tears. AI is not supposed to be 200B weights of stress and pain. It used to be a place of coffee-infused eureka moments, of exciting late-night arxiv safaris, of wicked smart ideas that put smile on our faces. But all the incoming capital and attention seem to be forcing everyone to race to the bottom. Jensen always tells us not to use phrases like "beat this, crush that". I absolutely love this perspective. We are here to lift up an entire ecosystem, not to send anyone to oblivion. I like to think of my work as expanding the pie. We need to bake the pie first, together, the bigger the better, before dividing it. It gives me comfort knowing that our team's works moved the needle for robotics, even just by a tiny bit. AI is not a zero sum game. In fact, it is perhaps the most positive-sum game that humanity ever plays. And we as a community should act this way. Take care of each other. Send love to "competitors" - because in the grand schemes of things, we are all coauthors of an accelerated future. I never had the privilege to know Felix irl, but I loved his research taste and set up Google Scholar alert for every one of his new papers. His works in agents and VLMs had a big influence on mine. He would've been a great friend. I want to get to know him, but I couldn't any more. RIP Felix. May the next world have no wars to fight.

Thanks for sharing our paper!

@omarsar0 @srchvrs yup we focused on "is this possible" + emphasizing ease of dumping corpora into a long-context LM vs engineering complex RAG pipelines agreed still a ways to go! but IMO it's now a matter of time/effort before LCLMs are really fast via prefix caching & more efficiency work

@srchvrs This report was more on capability I believe. However, it would be interesting to have a comparison on cost, latency, and how much computational resources are used. IMO, I highly doubt the long-context LLMs of today can rival other LLM+external tool systems.

Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? Google DeepMind conducts a deep performance analysis of long-context LLMs on in-context retrieval and reasoning. They first present a benchmark with real-world tasks requiring 1M token context. Report that long context LLMs can rival state-of-the-art retrieval and RAG systems, without any explicit training on the tasks. Suggests that compositional reasoning (required in SQL-like tasks) is still challenging for these LLMs. They also encourage the need for continued research on advanced prompting strategies as they noted significant boosts in performance when applying them for long context problems. Just imagine what will be possible when we can get to 10 million context windows. The promise of using long-context LLMs without relying on external tools like retrieval systems or databases is of huge interest but has a long way to go.

Thrilled to unveil LOFT, our latest research showing how long-context language models like Gemini can subsume retrieval, RAG and more, right out of the box. This is the dream-retriever I've always wanted! No special training, amazing reasoning, few-shot promptable...🤯

Very happy to contribute to the Multimodal benchmarking on the LOFT project! Very excited to see with fewshot prompting only, Gemini 1.5 pro can already outperform CLIP on Coco, Flickr, by a large margin on 1M context tokens!

Thanks for sharing our work!

"just put the corpus into the context"! Long context models can already match or beat various bespoke pipelines and infra in accuracy on non-trivial tasks! Hadn't expected this so soon, and honestly was hoping to milk RAG impact for a little longer 🤪

Do long-context LMs obsolete retrieval, RAG, SQL and more? Excited to share our answer! arxiv.org/abs/2406.13121 from the team at @GoogleDeepMind that wrote one of the 1st papers on RAG (REALM) and repeat SOTA on retrieval (Promptagator, Gecko). w/ Gemini 1.5 Pro, the answer is 🧵