philip

Excited to share FACTR 2, our work on external force estimation and force-aware policy learning for low-cost robot arms. In many dexterous manipulation tasks, success comes down to a few critical moments around contact. FACTR 2 uses learned force signals to better identify, prioritize, and learn from those moments. Really grateful to have had the opportunity to work with this team! Website: jasonjzliu.com/factr2/ Paper: arxiv.org/abs/2606.12406

@JasonJZLiu Incredible

Here are a few more tasks we trained using the system w/ @yangphiliphan

Force is arguably the most overlooked ingredient in modern robot learning. Introducing FACTR 2: it turns *any* commodity robot into a force-aware system with no force sensors required. Train a tiny force network in <1min with <10mins of data and drop it into any existing teleop pipelines: ✅ Free force sensing for both the robot and the operator arm ✅ Makes demos higher-quality → fewer of them needed. ✅ A new force-aware learning algorithm (FIRST) uses those recovered forces to figure out which parts of a demo actually matter, making learning data-efficient. ✅ Strong performance on complex tasks with fewer demos and even no pretraining! More details below.

New work on FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning: Paper: arxiv.org/abs/2606.12406 Web: jasonjzliu.com/factr2/ FACTR 2 shows that learned force signals can both enable force-feedback teleoperation on low-cost manipulators and improve behavior cloning (BC) policies for contact-rich tasks. It consists of two components: 1. Neural External Torque Estimation (NEXT): A lightweight model that infers external joint torques without dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): A training strategy that uses the learned force signal to identify and upsample task-critical moments. The key insight is simple: policy failures rarely occur in free space, they occur during brief pre-contact alignment and contact-rich interactions, where precise corrections matter most. Together, NEXT and FIRST bring force-aware teleoperation and robust long-horizon contact-rich policy learning to off-the-shelf robot arms, without requiring additional sensing hardware. See a more detailed thread by @JasonJZLiu.

Force sensing for low-cost robot arms — without adding force sensors. 🚀 Excited to share FACTR 2! 🚀 FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning. w/ @JasonJZLiu  @_tonytao_ 🧵(1/6)

What if some parts of a robot demonstration are more important than others? Most of a trajectory is free-space motion. But success or failure is often determined by a few critical moments around contact. In FACTR 2, we use force to find these moments and prioritize them for training. We find this helps policies learn better alignment and recovery behaviors, like the example below. w/ @StevenOh_ @JasonJZLiu 🧵(1/N)