Daniel Jiang

🚀New work on credit assignment in multi-step reasoning RL post-training🚀 Introducing Self-Reset Policy Optimization (SRPO): i) localize the first wrong reasoning step, ii) reset to that step, iii) learn from counterfactual continuations from there – no external supervision.🧵

🚀New work on credit assignment in multi-step reasoning RL post-training🚀 Introducing Self-Reset Policy Optimization (SRPO): i) localize the first wrong reasoning step, ii) reset to that step, iii) learn from counterfactual continuations from there – no external supervision.🧵

Today we're releasing MBABench 1.0: the first, spreadsheet-focused benchmark evaluating AI agents on building entire workbooks to professional standards. In the real-world, MBAs must produce complete, well-organized artifacts ranging from spreadsheets to presentations. These deliverables are then reviewed and revised by multiple stakeholders ranging from managing directors to client analysts, so beyond accuracy, they must meet various quality standards specific to the industry. We evaluated 10+ frontier models on end-to-end spreadsheet tasks, and found agents still fall short of the standards MBA grads are held to. (1/n)

🚀New work on credit assignment in multi-step reasoning RL post-training🚀 Introducing Self-Reset Policy Optimization (SRPO): i) localize the first wrong reasoning step, ii) reset to that step, iii) learn from counterfactual continuations from there – no external supervision.🧵

Self-correction for LLMs is usually done by critiquing an answer and regenerating the entire reasoning trace. We show targeted backtrack-and-resample works better: self-localize the first erroneous step, then sample a counterfactual. Repeat. [1\n]

Can LLMs self-improve reasoning without external supervision? Introducing MACA: RL for teaching agents how to better leverage multi-agent debate. No external labels—just copies of the model learning from each other. Improved collaboration, accuracy, and self-consistency. [1\n]