Hangoo Kang

Introducing TRACE: an end-to-end system for environment-specific agent self-improvement🚀 Outperforms direct RL on the environment, GEPA, and synthetic data approaches on τ²-Bench and ToolSandBox📈 Collab w/ @TarunSures41845, @JonSaadFalcon, @Azaliamirh. Details in thread👇

AI R&D agents look great in demos. They write code, fix bugs, and propose research-shaped ideas. But what do researchers actually spend their time doing? Fighting dependency conflicts, noisy metrics, and configs that I’m pretty sure worked 20 minutes ago. Can agents do that? 🧵

@blc_16 This is exactly the problem we tackle in our paper TRACE! TRACE contrasts successful vs. failed trajectories to identify specific capabilities the agent lacks on the task and generates synthetic capability-targeted RL environments with denser signals!

If you want to understand why RL struggles with long-horizon agent tasks, this is a good explanation. The core issue is that sparse rewards throw away most of the useful information in the trajectory. GEPA tries to learn from the trajectory itself, using reflection in text space, instead of only optimizing on the final reward. GEPA generates textual critiques of trajectories, proposes prompt edits, and then selects updates along a Pareto frontier between exploration and exploitation instead. Instead of collapsing everything into one reward number, it keeps more information about why a run failed and uses that to make legible changes. It’ll be interesting to see where this goes when people combine that kind of trajectory-level reflection with RL, using RL for optimization while preserving a much richer signal about why the agent succeeded or failed.

(5/n) Employing techniques like TRACE and RIFT will help us scale up our tool-use environments like Tau2-Infinity to be more robust, larger synthetic post-training datasets. arxiv.org/pdf/2604.05336 arxiv.org/pdf/2604.01375 TRACE authors: @hangoo_kang @TarunSures41845 @JonSaadFalcon @Azaliamirh RIFT authors: @qi_zhengyang @pham_derek @amanda_dsouza @ArminPCM @paroma_varma et al

Say hi to @OpenJarvisAI 👋 If you have issues, want to make a PR, or simply chat, just @OpenJarvisAI in a tweet! This account is itself an OpenJarvis instance: running 24/7 on an NVIDIA DGX Spark, triaging issues + PRs on the repo and serving as a personal assistant for the lab! For personal AI on personal devices, checkout: github.com/open-jarvis/Op… x.com/JonSaadFalcon/…

@kexun_zhang @TarunSures41845 @JonSaadFalcon @Azaliamirh We're actively adding more benchmarks!

@hangoo_kang @TarunSures41845 @JonSaadFalcon @Azaliamirh why not terminal bench / swe bench etc

Introducing TRACE: an end-to-end system for environment-specific agent self-improvement🚀 Outperforms direct RL on the environment, GEPA, and synthetic data approaches on τ²-Bench and ToolSandBox📈 Collab w/ @TarunSures41845, @JonSaadFalcon, @Azaliamirh. Details in thread👇

@Shahules786 @TarunSures41845 @JonSaadFalcon @Azaliamirh Sure, can you email me and cc the other authors?

@hangoo_kang @TarunSures41845 @JonSaadFalcon @Azaliamirh would love to chat, DM ?

@Shahules786 @TarunSures41845 @JonSaadFalcon @Azaliamirh Just fixed it, will be updated soon. Thanks for pointing it out!

@hangoo_kang @TarunSures41845 @JonSaadFalcon @Azaliamirh Also the paper link here #scaling-behavior" target="_blank" rel="nofollow noopener">scalingintelligence.stanford.edu/blogs/trace/#s… seems incorrect

Turns out we can get SOTA on agentic benchmarks with a simple test-time method! Excited to introduce LLM-as-a-Verifier. Test-time scaling is effective, but picking the "winner" among many candidates is the bottleneck. We introduce a way to extract a cleaner signal from the model: 1️⃣ Ask the LLM to rank results on a scale of 1-k 2️⃣ Use the log-probs of those rank tokens to calculate an expected score You can get a verification score in a single sampling pass per candidate pair. Blog: llm-as-a-verifier.notion.site Code: llm-as-a-verifier.github.io Led by @jackyk02 and in collaboration with a great team: @shululi256, @pranav_atreya, @liu_yuejiang, @drmapavone, @istoica05

Great work with @hangoo_kang , @JonSaadFalcon , and @Azaliamirh on a new system for environment-specific LLM agent self-improvement that trains the agent on the underlying capabilities it lacks 🚀

We thank Debangshu Banerjee, Tanvir Bhathal, Alex Bloom, Andy Dimnaku, Simon Guo, Sid Jha, Hermann Kumbong, Jacky Kwok, Andrew Shi, and Shayan Talaei for their feedback. Thanks to @PrimeIntellect, @LambdaAPI, @GoogleResearch, and @IBMResearch for compute that made this possible! 😊

And training on capability-targeted environments beats simply optimizing capability descriptions in the prompt. Learning the capability > describing the capability.

We release LLM-as-a-Verifier 🧠: A general-purpose verification framework that achieves SOTA 👑 on Terminal-Bench 2 (86.4%) and SWE-Bench Verified (77.8%) by scaling: - scoring granularity - repeated verification - criteria decomposition 📄 Blog & Code: llm-as-a-verifier.notion.site

Since the initial Intelligence-per-Watt release, we've extended the open-source profiling library to measure the intelligence efficiency of agentic workloads. Most recently, we wanted to calculate how many joules it takes to solve all the queries in TerminalBenchV2. This can help us better understand how much intelligence is delivered per joule and per watt on agentic workloads. Here’s what we found 🧵