David Dobáš

Last week, we ran 8 humanoid robots live at DJ Wukong’s Chinese New Year show. 5 on stage. 3 with VIPs. I ran the full pipeline, from mocap to sim to real deployment. Here’s what worked (and what didn’t).

@XueJia24682 This is a sped-up video of PHP Parkour php-parkour.github.io

✨🇨🇳This Unitree robot is doing parkour! Its moves are so cool.

@theo_michel42 Nano banana and manually edited. The first image was quite good, but then trying to edit with nano banana as well was pain 😁

@DDobas How did you do the graphics?

DreamZero vs LeWorldModel? DreamZero has very impressive results, but I still believe we will rather be predicting dynamics in compact latent spaces, and avoid expensive pixel prediction. That will allow to run models on-edge. LeWorldModel paper seems to have solved one of the biggest hurdles. So why is training latent world models hard? Naively, you would just encode your observations to a latent space, predict dynamics in latent space and compute loss between the precicted next latent and next embedded observation. But that would lead to collapse. The encoder and predictor can cheat, for example encoder will just predict latent all zeros for all observations, then predictor predicts next state zero as well. And since the embedding of the next state sould be zero as well, you have zero loss, the model is happy, but completely useless. So that’s why people spent a lot of time figuring out, how to regularize the latent space to avoid collapse. Several losses were proposed. LeWorldModel proposes only one additional loss, based on projecting latents to randomly chosen axis and forcing thise projections to follow Normal distribution. Ellegant and seems to work! What do you think — pixel dreaming or latent dynamics?

@DereWah Wow, so cool!

excited to announce SNAKE-WORLD, a world model trained entirely on EGO-SNAKE a world model fully trained on egocentric snake POV footage capable of simulating the natural habitat of vipers comment for a demo rollout

@chris_j_paxton Is it on GTC? Which booth?

This is the first time that I have seen Skild doing live demos in public! Demonstrating neural nets for precision manufacturing

Happy to have led the workshops at UFB Ghost Trials — covering video motion retargeting, training BeyondMimic in mjlab, latest papers like Sonic/BFM-Zero, plus team support, sim evals & real-robot deployments. Thrilled seeing more students get hands-on with humanoid training! 🚀🤖

Good job @NVIDIARobotics

Just deployed @nvidia Sonic on the G1 with @UFBots . Zero-shotted our custom move, no training needed. Everything is just one policy. Impressively stable.

@wind1266 @nvidia @UFBots But the cool thing is that you don't have to train a policy for the motion

@wind1266 @nvidia @UFBots Already retargeted motion. To generate a motion, you can use other libraries, for example video2robot github.com/AIM-Intelligen…

Also made our custom controller through websocket connection, so that we can control the robot from phone

Robots provided by @virtuals_io Performance prep: @ufb × @SentientOS_io

We only saw the stage a few days in advance. Luckily: carpet. When I saw confetti everywhere, I expected at least one slip. Somehow, they all held.

Last week, we ran 8 humanoid robots live at DJ Wukong’s Chinese New Year show. 5 on stage. 3 with VIPs. I ran the full pipeline, from mocap to sim to real deployment. Here’s what worked (and what didn’t).

@hansenlillemark How about accelerating objects or other nontrivial dynamics?

State of the art World Models still lack a unified world memory for representing and predicting dynamics out of their field of view. Why is that, and how can we fix it? Introducing Flow Equivariant World Models: models with memory capable of predicting out of view dynamics!🧵⬇️