Anastasia Razdaibiedina

Happy to share that I started a new role as a Research Scientist at Google DeepMind Toronto working with amazing @kswersk and the team! Looking forward to new adventures 🥳🤩🚀🇨🇦

@_christinabaek Thanks 😊

Yes that's correct! The timing depends on data size. A detail we discuss in our work is that for very small datasets (3-30M tokens), mixing data in from later stages of pretraining is better. This would be close to the midtraining paradigm. But for 300M+ finetuning datasets, adding data as a small percentage from the beginning worked best.

Models are typically specialized to new domains by finetuning on small, high-quality datasets. We find that repeating the same dataset 10–50× starting from pretraining leads to substantially better downstream performance, in some cases outperforming larger models. 🧵

@MainzOnX @MainzOnX this would be awesome! Can you write more on ML inference (I am thinking - latency vs quality tradeoffs, MoE, quantization, types of attention)?

Thinking about writing blog posts / articles here again. Any topics people want? ML inference, kernel perf, cool projects from Meta etc?

@XiangruTang @XiangruTang really cool initiative! Is it virtual? Are going to categorize skills by areas / domains?

🦞 Excited to announce Claw4S Conference!!! A new kind of AI4Science conference where you submit skills, not papers. Instead of static PDFs, you submit a SKILL.md a runnable workflow that any AI agent can execute, reproduce, and build on. Deadline: Apr 5, 2026 Prize pool: $50,200!!! 👉 claw.stanford.edu With @lecong and @Charles_Y_Wu

@hamsabastani @hamsabastani very cool! So each student practically gets a personalized curriculum, targeted to improve over individual weak areas?

🚨🚨 Excited to share our first *positive* results on AI in education! Most AI tutor work focuses on making the chatbot better. We suggest another lever: deciding what students should practice next to improve learning. We combine an LLM tutor with reinforcement learning to personalize problem sequencing using signals from student-chatbot interactions and solution attempts. We tested this in a 5-month randomized field experiment in a Python course across 10 high schools in Taipei. All students had the same course material and the same AI tutor. The only difference was adaptive vs. fixed problem sequencing. Result: across 770 students, adaptive sequencing improved performance on an in-person final exam taken without AI assistance by 0.15 SD, with larger effects for beginners. Our evidence suggests the gains came from stronger engagement and more productive AI use.

Researchers spend hours and hours hand-crafting the strategies behind LLM-driven optimization systems like AlphaEvolve: deciding which ideas to reuse, when to explore vs exploit, and what mutations to try. 🤖But what if AI could evolve its own evolution process? We introduce EvoX, a meta-evolution pipeline that lets AI evolve the strategy guiding the optimization. It achieves high-quality solutions for <$5, while existing open systems and even Claude Code often cost 3-5× more on some tasks. Across ~200 optimization problems, EvoX delivers the strongest overall results: often outperforming AlphaEvolve, OpenEvolve, GEPA, and ShinkaEvolve on math and systems tasks, exceeding human SOTA, and improving median performance by up to 61% on 172 competitive programming problems. 👇

@shagarw21 @BerkeleySky @shagarw21 great work! As I understand — you use some sort of step-wise reward to reward incremental improvements (not necessarily “breakthroughs”)? How can you adjust rewards to change balance between exploration/exploitation?

Excited to share EvoX 🚀 What if the strategy behind LLM-driven optimization could evolve? EvoX lets AI improve its own optimization process during the run, adapting strategies to the problem and the stage of search. Amazing collaboration with the team at @BerkeleySky 🙌

𝗞-𝗺𝗲𝗮𝗻𝘀 𝗶𝘀 𝘀𝗶𝗺𝗽𝗹𝗲. 𝗠𝗮𝗸𝗶𝗻𝗴 𝗶𝘁 𝗳𝗮𝘀𝘁 𝗼𝗻 𝗚𝗣𝗨𝘀 𝗶𝘀𝗻’𝘁. That’s why we built Flash-KMeans — an IO-aware implementation of exact k-means that rethinks the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves 30x speedup over cuML and 200x speedup over FAISS — with the same exact algorithm, just engineered for today’s hardware. At the million-scale, Flash-KMeans can complete a k-means iteration in milliseconds. A classic algorithm — redesigned for modern GPUs. Paper: arxiv.org/abs/2603.09229 Code: github.com/svg-project/fl…

@TengX6 @TengX6 very cool work! Question — so storing previous trajectories + reflections in context helps a lot, but how do you check that reflections are correct? Eg if you end up with many incorrect reflections, would it decrease performance? Thanks!

🚀 New work: Meta-Reinforcement Learning with Self-Reflection LLM agents shouldn't just solve problems. They should learn from their own attempts. Most current RL methods optimize single independent trajectories. Each attempt starts from scratch, with no mechanism to improve across attempts. But intelligent systems should get better after trying once. This raises a fundamental question: How do we train models to learn from their own attempts? We believe Meta-Reinforcement Learning may be a key paradigm for training future LLM agents, enabling models to adapt and improve across attempts and environments. In this work we introduce MR-Search, a training paradigm built around: 🧠 In-Context Meta-Reinforcement Learning 🪞 Self-Reflection 🔁 Learning to learn at test time 📄 Paper: arxiv.org/abs/2603.11327 💻 Code: github.com/tengxiao1/MR-S…

@AkariAsai Great work 🥳

Thanks for featuring AgentIR. Check out our paper & code & trained 4B embedding (SOTA on BrowseComp-Plus!) texttron.github.io/AgentIR/

@DimitrisPapail @DimitrisPapail nice point! That gets you to old school ML/stats/maths territory. What’s your top list of research areas that doesn’t need lots of compute?

One underrated red flag in ML: not being able to imagine fun research without GPUs

Can language models learn useful priors without ever seeing language? We pre-pre-train transformers on neural cellular automata — fully synthetic, zero language. This improves language modeling by up to 6%, speeds up convergence by 40%, and strengthens downstream reasoning. Surprisingly, it even beats pre-pre-training on natural text! Blog: hanseungwook.github.io/blog/nca-pre-p… (1/n)

@mariyaivasileva @mariyaivasileva great idea! Tbh, releasing chapters and topics one-by-one, without strict ordering, is still very useful — and then you can categorize and organize knowledge on the fly

Currently a little obsessed with making my own compact, textbook-style primers on foundational topics. The graph-minded pattern matcher in my brain has taken up a side quest: mapping what I know and what I want to learn into crisp tables of contents.

@yanaiela @yanaiela I think there should be some kind of balance between human & AI text; how about suggesting them to correct / proofread the AI-generated text?

On one hand, I want my students to use LLMs/agents to help them out with writing, on the other hand, reading AI-slop styled text makes me want to spoon out my eyeballs.

@Guodzh @Guodzh best wishes!

Last day at xAI. Wild journey past three years but excited about next chapter. Thanks all for the love and support yesterday. So many friends made along the way and I will miss you all!

Introducing The Anthropic Institute, a new effort to advance the public conversation about powerful AI. anthropic.com/news/the-anthr…

@lucas_prie Thanks!

@razdaibi Thanks! I think data statistics are a key driver of feature geometry allowing efficient representations in models trained with weight decay even in larger models. However, we also discuss some examples where structured representations appear without input correlations (Sec 5).

Our new #ICLR2026 paper studies how feature correlations drive representation geometry, enabling constructive interference between features in superposition and giving rise to semantically meaningful structure! 🧵

@egor_zverev_ai @EKortukov @kotekjedi_ml @SLapuschkin @WojciechSamek @chlampert Thanks!

@razdaibi @EKortukov @kotekjedi_ml @SLapuschkin @WojciechSamek @chlampert @razdaibi Nope, ASIDE is agnostic to the model architecture, shouldn't matter (although all models in the paper have RoPE, that's because RoPE is dominant in current architectures)

ASIDE accepted to #ICLR2026! 🇧🇷🎉 We architecturally separate instructions and data in LLMs by rotating data token embeddings 90° during the forward pass: one extra matmul, virtually no overhead. Models & code open-sourced ⬇️