Roberto

How do we turn a decent generalist VLA into a great multitask specialist? RL can turn experience into one fine-tuned policy. The challenge is doing it continuously as new tasks arrive: without forgetting old ones or limiting future learning. Surprisingly simple. Check this out.

Excited to share that DataMIL is accepted at ICLR. To celebrate we are also releasing the codebase: github.com/UT-Austin-RobI… You can now curate data for your robot within a couple of hours!

We are excited to release MoMaGen, a data generation method for multi-step bimanual mobile manipulation. MoMaGen turns 1 human-teleoped robot trajectory into 1000s of generated trajectories automatically.🚀 Website: momagen.github.io arXiv: arxiv.org/abs/2510.18316

A little late to this but excited to share that DataMIL won the best paper at the Data workshop at #CoRL! If you haven't already, check it out! 👇

🚨CoRL 2025 Best Poster Award 🏆 Paper Alert 🚨 ➡️Paper Title: Mash, Spread, Slice! Learning to Manipulate Object States via Visual Spatial Progress 🌟Few pointers from the paper 🎯Most robot manipulation focuses on changing the kinematic state of objects: picking, placing, opening, or rotating them. 🎯However, a wide range of real-world manipulation tasks involve a different class of object state change—such as mashing, spreading, or slicing—where the object’s physical and visual state evolve progressively without necessarily changing its position. 🎯Authors of this paper presented “SPARTA”, the first unified framework for the family of object state change manipulation tasks. 🎯Their key insight is that these tasks share a common structural pattern:they involve spatially-progressing, object-centric changes that can be represented as regions transitioning from an actionable to a transformed state. 🎯Building on this insight, SPARTA integrates spatially progressing object change segmentation maps, a visual skill to perceive actionable vs. transformed regions for specific object state change tasks, to generate a) structured policy observations that strip away appearance variability, and b) dense rewards that capture incremental progress over time. 🎯These are leveraged in two SPARTA policy variants:reinforcement learning for fine-grained control without demonstrations or simulation; and greedy control for fast, lightweight deployment. 🎯They validated “SPARTA” on a real robot for three challenging tasks across 10 diverse real-world objects, achieving significant improvements in training time and accuracy over sparse rewards and visual goal-conditioned baselines. 🎯Their results highlight progress-aware visual representations as a versatile foundation for the broader family of object state manipulation tasks. 🏢Organization: @UTAustin 🧙Paper Authors: Priyanka Mandikal, @JiahengHu1 , @ShivinDass , @sagnikmjr , @RobobertoMM , Kristen Grauman 📝 Read the Full Paper here: vision.cs.utexas.edu/projects/spart… 🗂️ Project Page: vision.cs.utexas.edu/projects/spart… 🎥 Be sure to watch the attached Technical Summary Video - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️QT and teach your network something new Follow me 👣, @NaveenManwani17 , for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements. #CoRL2025

Excited that SPARTA (vision.cs.utexas.edu/projects/spart…) won the best poster award at the CoRL RINO workshop! Big congrats to the project lead Priyanka, who worked so hard on this project, as well as to the rest of the co-authors @ShivinDass @sagnikmjr @RobobertoMM and Kristen!

📢 Call for Community Activities #AAAI2026 We invite submissions of proposals for including and open activities that help broaden community participation in the AI field. October 4: Submission Deadline October 18: Acceptance Notifications @RobobertoMM @marucabrera27 @RealAAAI

Simple but *so effective idea*! And it can be used with any feature data selector. Great work led by @sateeshk21 . Do not miss it at #CoRL2025 (Spotlight 4 & Poster 2 on Sept 29)!

Intelligent humanoids should have the ability to quickly adapt to new tasks by observing humans Why is such adaptability important? 🌍 Real-world diversity is hard to fully capture in advance 🧠 Adaptability is central to natural intelligence We present MimicDroid 👇 🌐 ut-austin-rpl.github.io/MimicDroid

Honored to give an Early Career Invited Talk at #IJCAI today. See you at 11:30am in room 520C!

It was time to improve our evaluations in robot learning! We introduce a methodology based on anonymous A/B testing: fairer, stronger, community-driven. Awesome work by @KarlPertsch @pranav_atreya @tonyh_lee and an incredible crowdsourcing team. Upload and test your model! 🚀

Imagine a future where robots are part of our daily lives — How can end users teach robots new tasks by directly showing them, just like teaching another person? 🧵👇

Meet Casper👻, a friendly robot sidekick who shadows your day, decodes your intents on the fly, and lends a hand while you stay in control! Instead of passively receiving commands, what if a robot actively sense what you need in the background, and step in when confident? (1/n)

@abhishekunique7 @JiahengHu1 Thanks, Abhishek!

Excellent work as usual from @JiahengHu1 and collaborators :)

Hell yea real world RL :)

🚨RL training for contact-rich tasks with a mobile manipulator IN THE REAL WORLD?!🤯 We're not crazy—just equipped with the right action space! SLAC learns a safe, effective action space via unsupervised RL in sim, enabling real-world RL training in minutes. Check it out!🚀

Excited to be in ATL for #ICRA2025 to present 🔥FLaRe: fine-tuning large transformer policies with #RL, 15:25 Tuesday @ room 410! I will also be attending the 📷Doctoral Consortium on Monday to talk about my research on self-improving robots. Happy to meet old and new friends!

Loved working on this with our MIT/Stanford/OpenAI collaborators. It brings "The Bitter Lesson" to data curation: skip the hand-tuned heuristics (visual similarity, motion...) and let the data speak for itself! Datamodels is a fascinating framework 🤯

Extremely excited to announce that I will be joining @UTAustin @UTCompSci in August 2025 as an Assistant Professor! 🎉 I’m looking forward to continuing to develop AI agents that interact/communicate with people, each other, and the multimodal world. I’ll be recruiting PhD students for Fall 2026 across a range of connected topics (details: esteng.github.io) and plan on recruiting interns for Fall 2025 as well. A huge shoutout to my mentors who have supported and shaped my research! Especially grateful to my postdoc advisor @mohitban47 for helping me grow along the whole spectrum of PI skills, and my PhD advisor @ben_vandurme for shaping my trajectory as a researcher, and of course the amazing students/collaborators from @uncnlp and @jhuclsp 🙏

So happy for @JiahengHu1 ! He has been rocking it, with outstanding work that pushes the limits of what robot learning can achieve in mobile manipulation and other domains. And one of my first Ph.D. students! Congratulations! 🦾🦾🦾🦾

✨Super excited to share what the team has been working on! ♊️🤖 Gemini Robotics is a family of frontier models that are dexterous, interactive, and general. It builds on top of Gemini's world understanding, enhancing it's spatial/embodied reasoning, and producing robot actions.