Billy Yan

Learning from human data requires human-like hardware. Humans use their wrists constantly, but table-top manipulators lack this flexibility. We build upon RUKA and introduce RUKA-v2: a tendon-driven hand with a 2-DOF wrist and finger abduction/adduction 👋✌️

Teleoperation was pioneered ~1950 to remotely handle radioactive material. When we use it today to collect robot trajectories for BC, it is still clumsy. Surely, there is a better way! (Hint: human video, RL in sim).youtube.com/watch?v=Iihxza…

✨ Meet YOR: Open-Source Bimanual Mobile Manipulator from @nyuniversity Fully open-source mobile manipulator with dual 6-DoF PiPER arms by AgileX Robotics, BOM cost only ~$10k! 🌐 yourownrobot.ai #Robotics #OpenSource #AgileXRobotics #PiPER #NYU

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment. pointbridge3d.github.io

Introducing YOR. Balancing budget and functionality for a capable mobile robot is always a challenge. To give researchers and hobbyists more options, we built our own open-source one for ~$10k.

Why buy a robot when you can build your own? Meet YOR, our new open-source bimanual mobile manipulator robot – built for researchers and hackers alike for only ~$10k. 🧵👇

We don't need the name of an object to pick it up; we simply need to know where it is and what it looks like. Introducing Contact-Anchored Policies (CAPs): instead of language, we explicitly condition on contacts. Our policy learns object pickup with only 16 hours of data! 🧵

Best ideas are often the simplest in hindsight. Meet Contact-Anchored Policies (CAP)🧢: by conditioning policies on physical contact (vs language) we achieve env & embodiment generalization with super low resources. This policy ⬇️ learned to pick from scratch w/ 16 hrs of data 🧵

We just released AINA, a framework for learning robot policies from Aria 2 demos, and are now open-sourcing the code: github.com/facebookresear…. It includes: ✅ Aria 2 data processing into 3D observations like shown ✅Training of point-based policies ✅Calibration Give it a try!

Dexterous manipulation by directly observing humans - a dream in AI for decades - is hard due to visual and embodiment gaps. With simple yet powerful hardware - Aria 2 glasses 👓 - and our new work AINA 🪞, we are now one significant step closer to achieving this dream.

When @anyazorin and @irmakkguzey open-sourced the RUKA Hand (a low-cost robotic hand) earlier this year, people kept asking us how to get one. Open hardware isn’t as easy to share as code. So we’re releasing an off-the-shelf RUKA, in collaboration with @WowRobo and @zhazhali01.

I gave a Early Career talk at CoRL 2025 in Seoul last week, where I talked about my observations from the past decade in robot learning along with where the field is headed for the next decade. In summary, the future of robot learning needs: (1) Data beyond teleop: We are never going to reach the scale of LLM / VLM data by tele-operating robots. Need to leverage consumer hardware already in people's hands (e.g. iPhones) and emerging devices (e.g. Smartglasses). (2) Observations beyond vision: The hard problem in robotics is dexterity. Dexterity is all about moving objects intricately through contact. The sense of touch is critical for this. Vision can help you acquire objects, but anything more complex will need touch. (3) Reasoning beyond reactivity: The biggest wins in robot learning have been in reactive policies (both manipulation and locomotion). But the class of models that got us here are generally feed-forward nets. Long-horizon reasoning needs the ability to predict future outcomes and manipulate them. Currently unclear what the right scalable architectures are here, but we are working on it. (thanks @zacinaction for the pic!)

How can we effectively leverage human videos for robot learning by bridging the inherent embodiment gap? We introduce UniSkill, a universal skill representation, a scalable method for learning cross-embodiment skill representations from large-scale in-the-wild video data. 1/n

It is difficult to get robots to be both precise and general. We just released a new technique for precise manipulation that achieves millimeter-level precision while being robust to large visual variations. The key is a careful combination of visuo-tactile learning and RL. 🧵👇

🚀 With minimal data and a straightforward training setup, our VisualTactile Local Policy (ViTaL) fuses egocentric vision + tactile feedback to achieve millimeter-level precision & zero-shot generalization! 🤖✨ Details ▶️ vitalprecise.github.io

Generalization needs data. But data collection is hard for precise tasks like plugging USBs, swiping cards, inserting plugs, and keying locks. Introducing robust, precise VisuoTactile Local (ViTaL) policies: >90% success rates from just 30 demos and 45 min of real-world RL.🧶⬇️

We have developed a new tactile sensor, called e-Flesh, with a simple working principle: measure deformations in 3D printable microstructures. Now all you need to make tactile sensors is a 3D printer, magnets, and magnetometers! 🧵

Tactile sensing is gaining traction, but slowly. Why? Because integration remains difficult. But what if adding touch sensors to your robot was as easy as hitting “print”? Introducing eFlesh: a 3D-printable, customizable tactile sensor. Shape it. Size it. Print it. 🧶👇

Making touch sensors has never been easier! Excited to present eFlesh, a 3D printable tactile sensor that aims to democratize robotic touch. All you need to make your own eFlesh is a 3D printer, some magnets and a magnetometer. See thread 👇and visit e-flesh.com

Everyday human data is robotics’ answer to internet-scale tokens. But how can robots learn to feel—just from videos?📹 Introducing FeelTheForce (FTF): force-sensitive manipulation policies learned from natural human interactions🖐️🤖 👉 feel-the-force-ftf.github.io 1/n