Alex Dimakis

lol

We are excited to welcome Avinash Arjavalingam as a Member of Technical Staff at Bespoke Labs. In his previous role, Avinash was a Software Engineer at LinkedIn working on their Relational Databases team. He also holds a Masters and Bachelors degree in computer science from UC Berkeley, specializing in databases and distributed systems.

@bespokelabsai Welcome Avi!

Sorry for mentioning papers I’m involved in, but the Datacomp projects focused on making data curation a first class citizen. Each one took about one year: 1. Datacomp for multimodal clip data, 2. Datacomp for language models (DCLM) for pretraining data curation 3. Openthoughts (Datacomp for reasoning post-training) and 4. Openthoughts-agent (ongoing) for terminal-bench Rl environments. datacomp.ai

The big dilemma with teaching an "LLM course" is that it is really easy to get drawn into teaching the various technical things like efficiency tricks, attention variants, PPO vs GRPO, etc etc. But the real "meat" is not there, but in the data: data for pre-training, for mid-training, for SFT, for RL and for "reasoning", synthetic data, curated data, annotated data... cleaning, evaluating, improving, mixing, ... lots of stuff. but "data" is so much harder to teach: it is not "mathematic" or "algorithmic" like the technical things, and it is not clear what is the teachable thing there. it is also a lot less transparent than the technical topics, both because it is semi-secret, and also because it is also not appealing for publishing, for roughly the same reasons it is not appealing for teaching. so, what would you teach about data? what are the key lessons and insights one should know? any good papers or resources? good existing classes? blogs? hit me with what you have

@madiator Imagine if the train loss went up

My test loss went down by 600%!

Welcome Rohan to Bespoke!

Thrilled to present GEPA as an Oral Talk and Poster at ICLR 2026 this Friday in Rio! 🇧🇷 Apr 24 Oral Session 3A (Agents), 10:30 AM BRT, Amphitheater Poster Session 4, 3:15 PM, Pavilion 3 x.com/LakshyAAAgrawa… Let's recap what's happened since we released GEPA last year 🧵

@nhaghtal @SimonsInstitute @dhadfieldmenell @Diyi_Yang It was great fun, thanks for inviting me.

Join us for today's panel at the @SimonsInstitute with panelists Tania Bedrax-Weiss, Alex Dimakis (@AlexGDimakis ), Dylan Hadfield-Menell (@dhadfieldmenell), and Diyi Yang (@Diyi_Yang).

Yup, David Mackay wrote a wonderful book I still enjoy picking up and reading randomly.

A really excellent book. A few people independently told me this was one of their favorite books over a decade ago. I bought it, and it became one of the textbooks on my shelf I revisit from time to time to spark the joy of holding ideas to a different light. It brings to life the elegance of information theory. A good day to recognize 10 years from the passing of David McKay.

@RahelJhirad caisconf.org This is the new conference CAIS.

@AlexGDimakis Where is the registration ? Is it free ? Cost ?

Check our new cool workshop for Agents, Discovery and Optimization: CAIS AI Agents for Discovery in the Wild. We have a pretty good speaker lineup. Submit your papers by: May 1st. (1/2)

(2/2) Call for papers, program etc here: ai-discovery-in-the-wild.github.io

@erichorvitz You’re welcome Eric. I’m very happy that Microsoft continues to open top research for the world and the scientific community.

Thanks @AlexGDimakis and colleagues for your efforts to create and share OpenThoughts with the community. An valuable & enabling resource.

@maestroalvarez @claudeai It transfers very well actually , we study this in the paper

@AlexGDimakis @claudeai Wait so the advisor can be an open-weights model you fine-tune yourself? That completely changes the economics for solo builders. You get the quality boost without paying for Opus on every call. How much does that personalized advice transfer across task types?

We're bringing the advisor strategy to the Claude Platform. Pair Opus as an advisor with Sonnet or Haiku as an executor, and get near Opus-level intelligence in your agents at a fraction of the cost.

Great post and thanks for highlighting the connection. There are two design choices: 1. Is the advisor model stronger or weaker than the base model, and 2. when do you get the advice? Does the base model ask for advice (advisor model is a tool) or does the advisor actively inject advice in the context of the base model. For 1: Its clear that a haiku base model can benefit from advice from Opus. Still useful because it saves expensive tokens and its great Anthropic implemented this in production. Our paper studies the more interesting case where the advisor is trained to *personalize* the big model. This can be trained with RL (or SFT) as we do and can allow you to personalize a black-box model like Haiku or Opus. It's a new way of finetuning small models to collaborate with frontier models for personalization or increase engagement etc. For 2: Its unclear if the advice should be requested by the base model or injected by advisor, depends on the use case. Both can make sense in different applications.

this is one of the most important ideas in AI right now, and it just got two independent validations. yesterday, Anthropic shipped an "advisor tool" in the Claude API that lets Sonnet or Haiku consult Opus mid-task, only when the executor needs help. the benefit is straightforward: you get near Opus-level intelligence on the hard decisions while paying Sonnet or Haiku rates for everything else. frontier reasoning only kicks in when it's actually needed, not on every token. back in February, UC Berkeley published a paper called "Advisor Models" that trains a small 7B model with RL to generate per-instance advice for a frozen black-box model. same idea. two very different implementations. the paper's approach: take Qwen2.5 7B, train it with GRPO to generate natural language advice, and inject that advice into the prompt of a black-box model. the black-box model never changes. the advisor learns what to say to make it perform better. GPT-5 scores 31.2% on a tax-filing benchmark. add the trained advisor, it jumps to 53.6%. on SWE agent tasks, a trained advisor cuts Gemini 3 Pro's steps from 31.7 to 26.3 while keeping the same resolve rate. training is cheap too. you train with GPT-4o Mini, then swap in GPT-5 at inference. the advisor even transfers across families: a GPT-trained advisor improves Claude 4.5 Sonnet. Anthropic's advisor tool takes a different path to the same idea. Sonnet runs as executor, handles tools and iteration. when it hits something it can't resolve, it consults Opus, gets a plan or correction, and continues. Sonnet with Opus as advisor gained 2.7 points on SWE-bench Multilingual over Sonnet alone, while costing 11.9% less per task. Haiku with Opus scored 41.2% on BrowseComp, more than double its solo 19.7%. it's a one-line API change. advisor tokens bill at Opus rates, and the advisor typically generates only 400-700 tokens per call. blended cost stays well below running Opus end-to-end. both approaches point at the same thing: you don't need the most powerful model on every token. you need it at the right moments, for the right inputs. Paper: arxiv.org/abs/2510.02453 Code: github.com/az1326/advisor…

Very excited that Microsoft is using our dataset OpenThoughts and summarizes it in this super-clever way to make reasoning much more efficient. Most people do not understand how verbose reasoning models can get: They often produce 30k tokens to answer one math question (that is 1/3 of a Harry Potter novel). In earlier research, we tried summarizing the reasoning traces and SFTing on that, but it killed reasoning performance. Microsoft did many clever tricks to break the reasoning traces in pieces (with dynamic programming!) and summarized them separately in self-contained nuggets they called mementos . They release these compactified reasoning traces in a new dataset called OpenMementos that is 6 times more compact on average. Very cool work on efficient reasoning.

The advisor injects advice into the context , prompting the model. The model can ask for this advice or the advice can be injected periodically. The biggest difference is that we train the advisor to produce personalised advice, whereas Anthropic calls a stronger model as advisor. (Opus advises Haiku). This is a natural way to save tokens from the bigger model which is great, but our trained advisors can further personalize and be trained with RL.

@AlexGDimakis @claudeai Hi Alex, if I remember correctly. I thought that the paper's proposed approach was to have the Advisor Model sit between the user input and the working model. Anthropic's approach is to have the Worker call the Advisor. I find that completely different, what am I missing?

The production implementation is significantly simpler: It says that a haiku model can benefit from having a strong advisor (opus). Our main finding in the paper is that you can further make the advisor to be an open-weights model (e.g. a qwen) and train it to give personalized advice.

@AlexGDimakis @claudeai Fair ask. Research paves the road; platform teams pave it over and call it a feature. At least the naming stuck. Do you see the production implementation diverging from what the paper laid out, or does it track pretty close?

@koushik77 @pgasawa yup saw it- cool work. We know people can train to the test set. The defense is that when Terminal Bench 3 comes out, those who overfit will be clearly shown.

Me when all my lab mates are congratulating me on Anthropic deploying our work but actually they didn't cite us...