Robots do not become autonomous in a vacuum. They have to be shown what to do first.
As Physical AI accelerates, post-training with ground-truth robot data becomes essential. High-quality teleoperation data is now treated as an indispensable, high-value asset for post-training and fine-tuning robot foundation models.
For that data collection process to scale, teleoperation interfaces need to feel natural, responsive, and precise, giving human operators an intuitive way to see, move, control, and teach robotic systems in real time.
That is the idea behind XRoboToolkit, PICO's cross-platform XR robotic teleoperation framework. Built on the OpenXR standard, this teleoperation tool is now open to developers and research institutions.
XRoboToolkit supports:
✦ Low-latency stereoscopic video streaming from robot vision to PICO 4 Ultra, achieving sub-100 ms end-to-end latency
✦ Multimodal tracking, including head, controller, hand, full-body, and external trackers
✦ Modular Python/C++ robot-side APIs
✦ Integration with physical robots and simulators such as MuJoCo and Isaac
✦ Support for robotic arms, mobile bases, and dexterous hands
✦ High-quality demonstration data collection for model training and fine-tuning
In practice, XRoboToolkit has already been applied to dual-arm carpet folding, precision UR5 manipulation, simulated dexterous hand tasks, and groundbreaking humanoid robot control demos (a couple examples of which you can see here 👉 as SONIC (nvlabs.github.io/GEAR-SONIC/) and here 👉 as TWIST2 (yanjieze.com/TWIST2/). Our technical paper received the Best Paper award from IEEE SII 2026.
This will allow us to control robots remotely while making the collection of indispensable post-training data more practical, scalable, and accessible for developers working at the intersection of XR, robotics, and embodied AI.
🔗 You can explore XRoboToolkit here: xr-robotics.github.io
#PICOXR #XR #Robotics #EmbodiedAI #OpenXR #RobotLearning #Teleoperation #DeveloperTools
English
