Peter Chen me-retweet
Peter Chen
150 posts
Peter Chen
@peterxichen
Building general purpose robotics at Amazon FAR. Previously Covariant CEO and Co-Founder, @OpenAI, @UCBerkeley PhD.
Bergabung Aralık 2017
2.5K Mengikuti4K Pengikut
Peter Chen me-retweet
Peter Chen me-retweet
New project! Flow Policy Gradients for Robot Control
tldr; a simple online RL recipe for training and fine-tuning flow policies for robots
co-led w/ @redstone_hong: hongsukchoi.github.io/fpo-control
English
Peter Chen me-retweet
We introduce a framework that enables robust, long-horizon bi-directional locomotion over complex terrains, by effectively leveraging a single policy with dual-depth camera streams, without the need for LiDAR-based elevation maps.
Check out @Yuanhang__Zhang's tweet for architecture and implementation details, and the website for full, uncut rollouts.
Yuanhang Zhang@Yuanhang__Zhang
Robust humanoid perceptive locomotion is still underexplored. Especially when different cameras see different terrains, paths get narrow, and payloads disturb balance... Introduce RPL, tackling this with one unified policy: • Challenging terrains (slopes, stairs and stepping stones); • Multiple directions; • Payloads; Trained in sim. Validated long-horizon in the real world. Watch the robot walk it all🦿 Details below👇
English
Peter Chen me-retweet
Robust humanoid perceptive locomotion is still underexplored. Especially when different cameras see different terrains, paths get narrow, and payloads disturb balance...
Introduce RPL, tackling this with one unified policy:
• Challenging terrains (slopes, stairs and stepping stones);
• Multiple directions;
• Payloads;
Trained in sim. Validated long-horizon in the real world.
Watch the robot walk it all🦿
Details below👇
English
Peter Chen me-retweet
Tired of waiting hours for humanoids to learn to walk?
Our new technical report shows how to train sim-to-real humanoid locomotion in 15 minutes with FastSAC and FastTD3! The full pipeline is open-source in the newly released Holosoma codebase.
Thread 🧵
English
Peter Chen me-retweet
Excited to share this latest work from our team! Holosoma is now our go-to option for humanoid research at FAR, and we will continue to maintain it and add new capabilities in the future. We're also hiring!
Research: amazon.jobs/en/jobs/285057…
Software: amazon.jobs/en/jobs/304305…
Carlo Sferrazza@carlo_sferrazza
Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.
English
Peter Chen me-retweet
Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma.
To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.
English
Peter Chen me-retweet
Open-source: complete codebase covering multiple simulation backends, training, retargeting, and real-world inference. Infra built for humanoid, but also readily modified for quadruped (also included). Lots of infra gems/conveniences we rely on consistently. Hopefully equally helpful for others.
Carlo Sferrazza@carlo_sferrazza
Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.
English
Peter Chen me-retweet
Peter Chen me-retweet
Excited to introduce TWIST2, our next-generation humanoid data collection system. TWIST2 is portable (use anywhere, no MoCap), scalable (100+ demos in 15 mins), and holistic (unlock major whole-body human skills).
Fully open-sourced:
yanjieze.com/TWIST2
English
Peter Chen me-retweet
Why override µP? Because its core assumptions only hold very early in training!
In practice wide models quickly stop being more sensitive to weight updates than smaller models! This is caused by changes in the geometric alignment of updates and layer inputs over training. 🧵6/8
English
We investigated what enables muP's LR transfer across model sizes and found some new insights into large model optimization dynamics:
Atli Kosson@AtliKosson
The Maximal Update Parameterization (µP) allows LR transfer from small to large models, saving costly tuning. But why is independent weight decay (IWD) essential for it to work? We find µP stabilizes early training (like an LR warmup), but IWD takes over in the long term! 🧵
English
Peter Chen me-retweet
I've long wondered if we can make a humanoid robot do a 𝘄𝗮𝗹𝗹𝗳𝗹𝗶𝗽 - and we just made it happen by leveraging 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁 with BeyondMimic tracking!
This came after our original OmniRetarget experiments, with only minor tweaks to RL training: relaxing a termination threshold and removing one reward term.
The policy achieved a 𝟱/𝟱 success rate in our real-world experiments, showing the strength of high-quality, interaction-preserving motion retargeting combined with BeyondMimic’s minimal RL tracking.
Here is the updated arXiv: arxiv.org/abs/2509.26633 (In Sec. V. A)
Zhen Wu@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 omniretarget.github.io 🎥 1/9
English
Peter Chen me-retweet
Super exciting news!! Welcome Nikita + co and looking forward to working together!
Nikita Karaev@n_karaev
🚨Big news - our team is joining the Frontier AI and Robotics (FAR) lab at @amazon to keep building the future of robotics. Excited to be joining such an inspiring team and to work with @peterxichen, @pabbeel, @rocky_duan, @akanazawa and many others at FAR.
English
Peter Chen me-retweet
SOTA data generation from OmniRetarget + SOTA formulation from BeyondMimic = mind-blowing performance
Zhen Wu@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 omniretarget.github.io 🎥 1/9
English
Peter Chen me-retweet
Our grand finale: A complex, long-horizon dynamic sequence, all driven by a proprioceptive-only policy (no vision/LIDAR)! In this task, the robot carries a chair to a platform, uses it as a step to climb up, then leaps off and performs a parkour-style roll to absorb the landing. This pushes the boundaries of agile, human-like loco-manipulation!
7/9
English
Peter Chen me-retweet
Very excited to start sharing some of the work we have been doing at Amazon FAR. In this work we present OmniRetarget, which can generate high-quality interaction-preserving data from human motions for learning complex humanoid skills.
High-quality re-targeting really helps the reinforcement learning.
Why? The control policy optimization landscape is much nicer for (near)feasible trajectories than for trajectories with artifacts like (e.g.) foot skating or ground penetration.
Zhen Wu@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 omniretarget.github.io 🎥 1/9
English
Peter Chen me-retweet
High-quality motion reference data is key for humanoid skill learning 🤖🕺💃
A natural idea is to leverage human motions and “translate” them to humanoid motions, a process known as retargeting. For interaction-rich tasks such as scene interaction and loco-manipulation, retargeting is challenging: it must ensure motion consistency, smoothness, kinematic feasibility (no artifacts like penetration or foot skating), and scalability (one framework can handle thousands of motions).
Excited to release OmniRetarget — a scalable retargeting method with a 4-hour high-quality humanoid motion dataset for interaction-rich tasks. OmniRetarget takes an interaction-preserving perspective: we optimize Laplacian deformation between source and target interaction meshes while enforcing kinematic constraints, producing consistent, smooth, and feasible trajectories at scale. Even better, OmniRetarget can efficiently augment motions by varying terrains, objects, and initial poses.
This high-quality interaction-preserving retargeting enables a minimal RL setup to execute long-horizon (up to 30s) agile, interaction-rich skills. All tasks in the video share just 5 rewards, 4 domain randomization terms, and rely only on proprioception.
More details: omniretarget.github.io
Zhen Wu@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 omniretarget.github.io 🎥 1/9
English
"Robotics has a data problem" -- and I love our initial stab at answering this question by the amazing team @lujieyang98 @x_h_ucb @zhenkirito123 @akanazawa @pabbeel @carlo_sferrazza Karen @rocky_duan @GuanyaShi
English
New work from Amazon FAR: with high-quality data, humanoids can perform a wide range of agile whole-body motion and loco-manipulation tasks with a very minimal RL & sim2real recipe. Check out website omniretarget.github.io for the range of tasks that can be performed by the same method.
Zhen Wu@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 omniretarget.github.io 🎥 1/9
English