Learning Agents Research Group

[1/n] Just wrapped up 7 months interning with @pcastr at DeepMind and I'm so excited to share our work: arxiv.org/abs/2602.10324. TLDR: We used LLM-powered program synthesis to automatically model and discover differences between human and LLM strategic behavior

Proud of our latest Nature publication, led by the Sony AI ethics team!

After 6 years as director of Texas Robotics, I'm delighted to hand over the reigns to the capable hands of Prof. Jose Millan. I'm honored to continue supporting Jose in my new role of "Founding Director" as I turn my main focus to acting as Chair of the UT Computer Science Dept

(1/n) With over 1,300 citations, MBPO is often cited as proof that model based RL beats model free methods. In arxiv.org/pdf/2412.14312 we showed it often completely fails in DeepMind Control. In our new work, Fixing That Free Lunch (FTFL), we explain why and make it succeed.

UTCS is hiring for several positions this year. Please share with anyone who may be interested! cs.utexas.edu/faculty/recrui…

Humans learn new manipulation skills from examples and improve as they see more examples. How can we endow robots with the same ability? 🤖 🚀We introduce RoboSSM, scalable in-context imitation learning that enables robots to learn and improve at test time—robots can improve with more examples without returning to the GPU for fine-tuning. 🧵 Paper: arxiv.org/abs/2509.19658 Video: youtube.com/watch?v=YR4m21…

[1/4] 🚀 We’re excited to announce the v1 release of JaxAHT – a new library for Ad Hoc Teamwork (AHT) research, built with JAX for speed & scalability! Check it out 👉 larg.github.io/jax-aht #AI #MARL #ReinforcementLearning #JAX #AdHocTeamwork

[1/8] New social navigation paper + benchmark: SocialNav-SUB 🚶🤖 Recent work puts VLMs on robots for navigation, but can they really interpret scenes and extract key details for social navigation? 🔎 larg.github.io/socialnav-sub

Congrats to Jiaxun Cui, the 29th Ph.D. graduate from my lab, on the defense of her dissertation entitled "Communication and Generalization in Multi-Agent Learning". Pictured here with me and the rest of her thesis committee (Amy Zhang, Yuke Zhu, Sandeep Chinchali, Yuandong Tian)

Introducing VGC-Bench, a new plug-and-play benchmark made to support research efforts in the domain of Pokémon VGC! Read on for the open source code, the paper, and everything you need to dive in. [1/9]

If the communication among autonomous vehicles is done in natural language, in principle, human drivers could communicate with them as well. This paper lays the groundwork for that happening!

[1/n] Can LLM Agents learn to communicate and coordinate in natural language in driving scenarios 🚗 through self-play interactions? Our recent research shows the potential for such learning; interestingly, there is evidence that they develop concise protocols for collaboration! A distilled version of the agents’ model could generalize to multiple scenarios, communicate at 250 bytes per message, and make decisions within 500 ms while maintaining the original (large) model’s performance. 💬Natural Language Communication among Autonomous Agents ✨Multi-agent Gymnasium for Policy Learning 👀Partial Observation and Negotiation Tasks More videos & analysis project page: talking-vehicles.github.io arXiv: arxiv.org/abs/2505.18334 with @ChenTangMark, Jarrett Holtz, Janice Nguyen, @aleallievi, @HangQiu, @PeterStone_TX open to discussion & collaboration! @utlarg @texas_robotics #LLM #multiagent

Real-world RL, where robots learn directly from physical interactions, is extremely challenging — especially for high-DoF systems like mobile manipulators. 1⃣ Long-horizon tasks and large action spaces lead to difficult policy optimization. 2⃣ Real-world exploration with whole-body contact raises serious safety concerns. 🚀 Introducing SLAC, a framework that brings safety and efficiency to whole-body real-world RL. Paper: arxiv.org/abs/2506.04147 Video: youtube.com/watch?v=bj5Ghj… 🧵

And congrats to co-first-author Zhihan as well!

👏

Huge congratulations to @PeterStone_TX for receiving the ACM Allen Newell Award! 🎉 We are proud to celebrate this incredible recognition of his groundbreaking contributions to AI and computer science.👏 #ACM #AAAI #AllenNewellAward

Congratulations to my Ph.D. student, Jiaheng Hu - co-advised with Roberto Martin-Martin. cs.utexas.edu/news/2025/jiah…

Check out our work for further details. Paper: arxiv.org/abs/2411.19418 This work couldn’t have been possible without amazing collaborators: @harshit_sikchi @PeterStone_TX and @yayitsamyzhang @utlarg @lab_midi @texas_robotics @MLFoundations(4/n)

In multi-object env, why do most Unsupervised Skill Discovery methods fail to learn complex skills like tool use? Because they simply maximize state coverage. Introducing our solution SkiLD: Skill Discovery Guided by Factor Interactions (NeurIPS24) wangzizhao.github.io/SkiLD/

🚀Unsupervised RL can learn skills purely from reward-free interactions with an environment. But what form of skills can facilitate efficient downstream hierarchical learning? Introducing DUSDi: Disentangled Unsupervised Skill Discovery (NeurIPS24). jiahenghu.github.io/DUSDi-site 🧵