Steven Oh
30 posts
@StevenOh_
Incoming CS PhD @uchicago | MechEng @waseda_univ
🤖 How can we scale up humanoid robot learning? Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed! Website: vision-language-kinematics.github.io
1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: forceband-emg.github.io 📄 Paper: arxiv.org/abs/2606.26093 💻 Code: github.com/Bottle101/Forc… 🎥 Video: youtube.com/watch?v=Otw6uX… 🧵 1/n
Force is arguably the most overlooked ingredient in modern robot learning. Introducing FACTR 2: it turns *any* commodity robot into a force-aware system with no force sensors required. Train a tiny force network in <1min with <10mins of data and drop it into any existing teleop pipelines: ✅ Free force sensing for both the robot and the operator arm ✅ Makes demos higher-quality → fewer of them needed. ✅ A new force-aware learning algorithm (FIRST) uses those recovered forces to figure out which parts of a demo actually matter, making learning data-efficient. ✅ Strong performance on complex tasks with fewer demos and even no pretraining! More details below.
Force sensing for low-cost robot arms — without adding force sensors. 🚀 Excited to share FACTR 2! 🚀 FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning. w/ @JasonJZLiu @_tonytao_ 🧵(1/6)
Force sensing for low-cost robot arms — without adding force sensors. 🚀 Excited to share FACTR 2! 🚀 FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning. w/ @JasonJZLiu @_tonytao_ 🧵(1/6)
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors. We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies. FACTR 2 consists of: 1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training. w/ @StevenOh_ @_tonytao_ 🧵(1/N)
What if some parts of a robot demonstration are more important than others? Most of a trajectory is free-space motion. But success or failure is often determined by a few critical moments around contact. In FACTR 2, we use force to find these moments and prioritize them for training. We find this helps policies learn better alignment and recovery behaviors, like the example below. w/ @StevenOh_ @JasonJZLiu 🧵(1/N)
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors. We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies. FACTR 2 consists of: 1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training. w/ @StevenOh_ @_tonytao_ 🧵(1/N)
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors. We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies. FACTR 2 consists of: 1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training. w/ @StevenOh_ @_tonytao_ 🧵(1/N)
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors. We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies. FACTR 2 consists of: 1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training. w/ @StevenOh_ @_tonytao_ 🧵(1/N)
