DSPy retweetledi
Your RL algorithm makes *LLMs* smarter.
Our RL algorithm makes *LLM Training* smarter.
These are not the same
English
DSPy
3.2K posts
DSPy
@DSPyOSS
An open-source declarative framework for building modular AI software. Programming—not prompting—LLMs via higher-level abstractions & optimizers.
Katılım Nisan 2025
61 Takip Edilen13.8K Takipçiler
DSPy retweetledi
DSPy retweetledi
ICYMI: read the blog on Pedagogical RL
Instead of sampling blindly from your LLM, leverage the label used for RLVR! Learn to directly approximate the distribution of your LLM's plausible rollouts that are actually correct. Then sample from *that*!
noahziems.com/pedagogical-rl
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
If the model can’t already stumble into success or at least useful states, on-policy methods stall.
Instead, we argue that an ideal sampler would give our LLM its “nearest successes”: rollouts that are correct, but where every step makes sense to the LLM and thus can be learned
English
@MaximeRivest wow, one year of Maxime in this community :D, we have been so lucky to have you!
English
DSPy retweetledi
dspy still feels magical, no other AI engineering framework comes close to it, and their is so much to come that will improve it!
Maxime Rivest 🧙♂️🦙🐧@MaximeRivest
trying dspy this morning.. I was reading the docs and I didnt full get it. Mostly because I thought the example were missing pieces to run. BUT NO. its actually that short. It feels a bit magical.
English
DSPy retweetledi
Seeing @lateinteraction’s name means you’re about to read a banger (and probably will use the method at some point)
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
alright, two of the five are out since April 24 :D
funnily enough, i'm still somehow excited about five (3+2) again currently. y'all will find them really really nice.
Omar Khattab@lateinteraction
Being this excited about five rather unexpected research projects simultaneously is almost too painful. Assuming that we figure out how to sequence these releases, y’all are going to thoroughly love each of these.
English
DSPy retweetledi
Extremely excited about our recent work in Pedagogical RL. I’m optimistic approaches like this are going to completely shift how data collection is done for hard agentic tasks like coding
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
Great teachers craft demonstrations their students could have built themselves.
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
@lateinteraction constant factor, close enough :) veeeery nice approach, really excited to dig into it further!
English
DSPy retweetledi
Are we really using all the available information when we post-train RL on hard tasks? Maybe not. 🧠
New framework — Pedagogical RL: teaching models to teach themselves to sample trajectories that are both correct AND actually learnable.
Up to 40% gains over GRPO and OPD 🚀
👇
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
A good research is when something feels intuitive, yet somehow you didn't think of it.
Testing this on my runs.
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
In other words: Humans are teaching teacher models how to teach other models the way good human teachers teach other humans so we can make smarter models that can teach humans to be smarter.
Intuition: A good teacher model will not only lead to the right answer--it will do so following a sequence of steps that the student can follow. Teacher models are penalized for taking leaps that feel like they came out of nowhere.
More cool work out of @lateinteraction's CSAIL lab!
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
End the tyranny of on-policy algorithms in LLM post-training!
Maybe the key thing isn't whether your rollouts are purely "on-policy" or not, but the extent to which they’re pedagogically useful.
Early explorations into newer paradigms for RL by @SOURADIPCHAKR18* @NoahZiems*:
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
Our recent work on Pedagogical RL is out!
Souradip Chakraborty@SOURADIPCHAKR18
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? ⤵️ Pedagogical RL
English
DSPy retweetledi
🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them.
We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute?
⤵️ Pedagogical RL
English
DSPy retweetledi
Databricks is proud to be a Founding Gold Sponsor of @TheOfficialACM Conference on AI and Agentic Systems—the first ACM conference dedicated to compound AI and agentic systems, with our co-founder @matei_zaharia on the organizing committee.
Join us May 26–29 in San Jose for the premier event for rigorous, reproducible research in compound AI architectures, optimization, and deployment.
Register today: caisconf.org
English
DSPy retweetledi
In San Diego / UCSD today & tomorrow — would love to grab a coffee or say hi to anyone around 👋 please reach out!
English