Dharmesh Kakadia

Breaking: we release a fully synthetic generalist dataset for pretraining, SYNTH and two new SOTA reasoning models exclusively trained on it. Despite having seen only 200 billion tokens, Baguettotron is currently best-in-class in its size range.

Post-training is the way forward. For what it's worth, the original RAG paper actually finetunes the LLM. It's useful to post-train the models to learn how to use search. If you are doing agentic RAG, you should figure out how to do RL there. When OpenAI wants to build Deep Research agent, they post-train their models to become these agents, and not prompt them. So I am yearning for a future when everyone can do this, not just the labs. And also if you believe in the era of experience, it's really prescribing post-training as well (RL).

1/ Been asked by 6+ funds and my partners about RFT and environment creation startups. Here’s what I’m looking to invest in and why the first wave of startups here have gotten it wrong 👇

i'm increasingly convinced that "transformative ai" is going to look like an abundance of specialized models for everything from drug design to weather sims to robotics to supply chains, not one agent to rule them all. we're going to need a lot more ai researchers