Eric Malmi

Language models can't play chess, right?♟️ Excited to share our latest experiment that let's you play chess against a Gemini model!

come chat to Jakub Adamek, @anianruoss, and me at the poster

if you're at #icml2025, come check out our spotlight poster on "Mastering Board Games by External and Internal Planning with Language Models" ♟️ 📜: arxiv.org/abs/2412.12119 ⏲️: Wed 16 Jul 11 am - 1:30 pm PDT 📍: East Exhibition Hall A-B #E-2508 demo: goo.gle/ChessChamp

Poster Spotlight! 🔦 Mastering Board Games by External and Internal Planning with Language Models ♟️ arxiv.org/abs/2412.12119 On Wednesday (Poster Session 3 East) Presented by Jakub Adamek and @ericmalmi

🚨Breaking: New Gemini-2.5-Pro (06-05) takes the #1 spot across all Arenas again! 🥇 #1 in Text, Vision, WebDev 🥇 #1 in Hard, Coding, Math, Creative, Multi-turn, Instruction Following, and Long Queries categories Huge congrats @GoogleDeepMind!

thank you for the recognition @GaryMarcus! there's room for improvement, but I find it quite remarkable that an LLM learns to play creative sacrifices like this (best move according to Stockfish)

@GaryMarcus @RepresenterTh you're welcome to test the MAV model (w/o MCTS) at: goo.gle/ChessChamp a few things to note: * for now, comments come from a different model so they can be ungrounded * MAV can play chess960, Hex, Connect4, but the Gem only supports chess

@RepresenterTh thank you!

Has anyone found an LLM that can reliably play chess, without making illegal moves?

🚨Breaking: @GoogleDeepMind’s latest Gemini-2.5-Pro is now ranked #1 across all LMArena leaderboards 🏆 Highlights: - #1 in all text arenas (Coding, Style Control, Creative Writing, etc) - #1 on the Vision leaderboard with a ~70 pts lead! - #1 on WebDev Arena, surpassing Claude for the first time This is the first-ever sweep across text, vision, and WebDev by any model!🥇 Huge congrats to @GoogleDeepMind on this incredible breakthrough!

@LaatoSamuli @anianruoss Thanks, Samuli!

@ericmalmi @anianruoss You are living the dream! Congrats🥳

multiple long-time dreams coming true at once: ✅ give a talk at NeurIPS ♟️ play chess on a stage 🤡 make my international debut as a rapper thanks to the audience for a lively discussion that went on for a good hour after the talk and to my amazing co-presenters @anianruoss @weballergy @MatejJusup!

Think you can outsmart Gemini? We challenge you to a chess match! Play Gemini in a game of chess with our newest Gem: Chess champ. Explore different openings as you banter back and forth with Gemini. Available in the Gemini web app. ♟️Can you beat it? → goo.gle/ChessChamp

@Bayesprof Thanks for joining!

This was a great talk. Also really surprising that transformers can imitate “search”!

try playing against our no-search model at: gemini.google.com/gem/chess-cham… and check out the paper: deepmind.google/research/publi…

our work establishes new test-time scaling results for chess-playing LLMs ♟️📈 honestly, I think it's quite mind blowing that an LLM can learn to perform minimax tree search within a single model call and smoothly improve its Elo the more output tokens you give it 🤯

LLMs can play chess! In-context minimax search bootstrapped with values from Stockfish, implemented in Gemini. Paper: storage.googleapis.com/deepmind-media… Breadth 4, depth 2, you start running out of context window. Chess Elo improves with more test-time compute. Really cool work from @ericmalmi @GoogleDeepMind

Paper: deepmind.google/research/publi… Demo: gemini.google.com/gem/chess-cham…

if you're at #NeurIPS2024, want to learn how to make LLMs really good at chess and see a live demo, come and visit the @GoogleDeepMind booth tomorrow at 9:30 am!

@PreethiLahoti Haha, you know me :) This is actually a great example of Gemini's generalization capabilities (no, I did not produce training data for this use case 😁)!

@ericmalmi Effortlessly combining your two passions: AI chess and AI rappers :)

Language models can't play chess, right?♟️ Excited to share our latest experiment that let's you play chess against a Gemini model!

You can try it at: gemini.google.com/gem/chess-cham… (requires Gemini Advanced subscription but you can get 1 month for free). The experiment is powered by the MAV–small model from our new paper: deepmind.google/research/publi…

Our first talk at @IC2S2 today shows that publishing early on arXiv and tweeting leads to more citations in the 5 years from the initial publication. Excellent talk and work by @c_bagchi! Our (revised, soon to appear) paper is accepted to ICWSM'25: arxiv.org/abs/2401.11116