Michael Equi

We developed RECAP @physical_int to apply RL and interventions to π0.6, achieving high success rates and throughput on several challenging tasks! Watching these policies operate successfully for hours gives an appreciation for what the method can do

Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!

Using "rl tokens" lets us improve our policies on highly dexterous tasks within just a few hours. In some cases we even see our policy achieve super human performance through both improved speed and the use of emergent strategies!

We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory. Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇

General-purpose AI models are behind some of the most exciting applications we now can't live without. We envision that an analogous “physical intelligence layer” built with models like π0.6 will similarly spur a new wave of applications for the physical world. We’ve recently begun working with a handful of companies that have deployed their robots to do real-world, useful things. pi.website/blog/partner/?…

We got our robots to wash pans, clean windows, make peanut butter sandwiches, and more! Fine-tuning our latest model enables all of these tasks, and this has interesting implications for robotics, Moravec's paradox, and the future of large models in embodied AI. More below!

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

Treat yourself to a 4-hour timelapse of a robot assembling boxes off-site:

We developed a general recipe that allows VLAs to improve from experience. RL is back. (yes, this is 13 hours of coffee making)

And finally a shoutout to an incredible team that was not only critical for bringing this together but also made it a blast to work on! If you are interested in more details you can find them in our blog post and paper: pi.website/blog/pistar06

We show that on these challenging real-world tasks, more iterations continue improving the performance of π*0.6. This emphasizes the value of learning from online experience and points a direction for how we may be able to scale the deployment of robots in the real world

We developed RECAP @physical_int to apply RL and interventions to π0.6, achieving high success rates and throughput on several challenging tasks! Watching these policies operate successfully for hours gives an appreciation for what the method can do

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.

We just released results for our newest VLA from Physical Intelligence: π*0.6. This one is trained with RL, and it makes it quite a bit better: often doubles throughput, enables real-world tasks like folding real laundry and making espresso drinks at the office.

We've added pi-05 to the openpi repo: pi05-base, pi05-droid, pi05-libero. Also added PyTorch training code!🔥 Instructions and code here: github.com/Physical-Intel… This is an updated version of the model we showed cleaning kitchens and bedrooms in April: pi.website/blog/pi05

We’re releasing the RoboArena today!🤖🦾 Fair & scalable evaluation is a major bottleneck for research on generalist policies. We’re hoping that RoboArena can help! We provide data, model code & sim evals for debugging! Submit your policies today and join the leaderboard! :) 🧵

Our models need to run in real time on real robots, but inference with big VLAs takes a long time. We developed Real-Time Action Chunking (RTC) to enable real-time inference with flow matching for the π0 and π0.5 VLAs! More in the thread👇

We figured out how to train VLAs with diffusion outputs much faster (7.5x faster), inheriting better language following from the VLM, and leading to better results. The key: protect the VLM backbone during training with knowledge insulation. Let’s talk about what we learned👇

We cover these improvements along with multiple others in our paper and blog post. The blog also provides many more examples demonstrating π-0.5 doing a variety of tasks, all in unseen environments! blog: https://www.π.com/blog/pi05 paper: https://www.π.com/download/pi05.pdf

One surprising insight is that co-training on the HL objective significantly improves performance even without the hierarchical inference scheme. We call this implicit HL

@physical_int we trained π-0.5, a model that can do tasks in homes never seen in training! Here is one video of π-0.5 completing a long-horizon task in an unseen kitchen while I provide natural language instructions. A short thread 🧵