Eliot Xing

Newton 1.0 is now generally available. 🙌 Take robot learning to the next level with: 🤖 Stable Articulated & Complex Mechanism Simulation – accurate, reliable machine modeling. 🖐️ High-Fidelity Hydroelastic Contact Modeling – realistic soft contact and touch-based interactions. 🧵 Deformable Body Simulation – simulate cables, cloth, rubber, and other elastic materials with VBD. ⚡ Accelerated Robot Learning at Scale – seamless integration with open simulation and learning frameworks, NVIDIA Isaac Sim and Isaac Lab for scalable workflows. Learn how to integrate this open-source physics engine into your workflow: nvda.ws/3NGTzUo #NVIDIAGTC

check out the latest from @TairanHe99 and co. on visual sim2real humanoid loco-manipulation!! really enjoyed the timeline documenting the sim2real journey

amazing work from @ChaoyiPan on physics-based retargeting for dexterous robot hands and humanoids that runs out of box on multiple physics simulators!

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: nvlabs.github.io/SONIC/ #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

What if robots could improve themselves by learning from their own failures in the real-world? Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks. PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel. Quick 🧵

The laws of physics apply everywhere. ⚛️ Co-developed by NVIDIA, @GoogleDeepMind and Disney Research, Walt Disney Imagineering, the new Newton Beta, managed by the @linuxfoundation, now also runs with Isaac Lab and with MuJoCo Warp, which delivers warp speed for robot learning. Learn how to train a quadruped and simulate multiphysics in our tech blog. 🔗 nvda.ws/4gP2woL #CoRL2025

cool work from @Jsphamigo! decouple first-order RL with data from nondifferentiable simulation + gradients from a learned differentiable dynamics model, for large sample efficiency gains and quadruped sim2real

📣MuJoCo announcement 📣 Thrilled to share that @GoogleDeepMind has unveiled MuJoCo-Warp at @nvidia's #GTC25! 🚀 We've expanded our open-source MuJoCo simulator with MuJoCo-Warp, leveraging NVIDIA’s Warp framework for incredible acceleration. This marks a significant step in making high-performance simulation more accessible.

exciting news on (differentiable?) simulation using Warp

🚀 We've already extended MOE to various dexterity projects including dynamic pen spinning (arxiv.org/abs/2411.12734) and learning in-hand manipulation skills from demonstration (arxiv.org/abs/2503.01078). 🧵6/7

Safety is hard to ensure in human-robot interaction with rigid manipulators. We present a soft robot system to provide safer interaction for hair care. @UksangYoo has also already used his soft manipulator for more dexterous tasks like pen spinning, check out his full thread!

@aaronwetzler no, total fps for forward sim is 1.4k with 32 envs, see appendix F.5 of the paper!

@etaoxing Got it. So for 32 envs it would be 6-7 fps?

RL is notoriously sample inefficient. How can we scale RL on tasks much slower to simulate than rigid body physics, such as soft bodies? In our #ICLR2025 spotlight, we introduce both a new first-order RL algorithm, SAPO, and differentiable simulation platform, Rewarped. 1/n

@Rawlala1 Thanks! We use physics solvers in Warp, with a custom MPM solver for some of the soft bodies.

@etaoxing is this physx under the hood ? very nice work

@Stone_Tao thanks Stone!! open source >>>

@etaoxing very cool! lots to learn here

@aaronwetzler thanks! fps varies depending on tasks, we use a few thousand particles each x 32 envs for mpm simulation. 1 env for handflip runs at 210 fps.

@etaoxing Looks great! What fps did this run when you recorded? And how many facets/particles/triangles for each sim?

@Hongyu_Lii thanks Hongyu!!

@etaoxing Awesome work. Congrats 👏

@YouJiacheng thanks for the kind words on our work!

Very Cool! Thank you for bringing us this HUGE win!!!

RL for deformables!

@milesmacklin Thanks Miles! This work would not be possible without Warp!

Congratulations @etaoxing! This is a super cool use of Warp and differentiable simulation!