Zu Wang

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.

1/ Progress in robot locomotion scaled with simulation, but touch was left behind. Our work, Tactile Genesis, unlocks sim for dexterous manipulation. Using sim to ablate tactile types and models, we can quickly converge to better tactile hardware and force-informed policies.

Feels like the non-steerability here is really just the policy overfitting to image obs -> action shortcuts because pretraining data isn't large/diverse enough; the fix should be scale and proper language labels at pretraining, not post-hoc steering data + finetuning. Also the steerability score depends entirely on which tasks you pick, so it's a pretty arbitrary number.

It’s becoming embarrassingly obvious that dexterous hands are the path to physical AGI. If you’re not working on hands, ngmi.

@flooburrito @clayhaight vibe driving all day

Sorry for cutting you off this morning @clayhaight

If I cut you off in traffic, I’m 99% sure it was FSD

I think it's a good investment of time for anyone working on robot policy learning to understand where our algorithms came to be; even when you can just use off-the-shelf packages to get a robot going in no time. Also, we have many practical tips that should be relevant for imitation-learning based manipulation research: supervised-robot-learning.github.io Thank you for the kind words, @zuwang95!

@AlexiGlad spot on

ive finally matured and realized that we need a city with the culture of NYC and the weather of SF/san diego

@iyanmoonyang supervised-robot-learning.github.io One of the best intros I always recommend ppl new to robot learning to watch. @notmahi is my robot learning 101 professor.

anyone willing to give me a robotics ML crash course over lunch. Being surrounded by amazing researchers is making me very curious but I fear I cannot disturb them all the time. I’ll pay for your food?

@shyyang what a visionary

It will be robots

Consider this a time capsule. Let's revisit in 5 years and see if it's been solved. cc @alpercanbe

@alpercanbe And thats the @alpercanbe I like.

@zuwang95 a bit spicy 🌶️

learning from third person video is solving a problem robots don’t have humans only do it because we can’t pipe sensory experience across brains. robots have no such limit. put on some cameras + gloves, and a robot can replay exactly what you see and feel

@alpercanbe Great take

btw, ego datasets that are being curated today will dwarf the third person videos one can scrape on the internet

@alpercanbe @SanghyunSon That I agree.

@zuwang95 @SanghyunSon in person evals hit different tho

There are a lot of new joiners and interns at Genesis this summer, so many that @Zhenjia_Xu suggested I might need to move to a different desk away from the robot corner, since most of the time I only use the robots for evals and don’t need to be there full-time. I still vividly recall around this time last year, it was just the two of us sitting there, babysitting a pair of simple bimanual gripper setups parked right in front of our desks. It’s crazy how much has changed since then. Our first working policies were trained on a single 5090. Today, we have two big clusters at our disposal. We spent countless hours experimenting with different data collection setups and gathering data ourselves. Now, collecting data is often as simple as submitting a ticket in Slack, and the requested data shows up in an S3 bucket right on schedule. Not to mention we now have some of the most advanced robot setups and even our own robot! I remember telling Xian the first time we met that the thing I care about most is being able to continuously learn new things as quickly as possible. Over the past year, Genesis has been exactly that kind of environment. I hope this momentum continues.

Excited to share Do as I Do! We turn everyday human videos into physically consistent robot data that can be directly executed in the real world. This was a fun collaboration with @bhawna_paliwal_ and @willjhliang, with lots of moving parts. More details in Mahi's thread below👇

@HarithejaE @bhawna_paliwal_ @willjhliang Congrats on the release!

Meta HQ right now.

@perryzjia agree

The best evals have both objective and subjective elements

On stage at @VivaTech, @theo_gervet co-founder of @gs_ai_ . « Humanoid robots don't need to look human. Meet Eno, our first general-purpose robot. Not a machine pretending to be human, but intelligence given a body. We’re building a future where robots are ready to help. » 🇫🇷🇺🇸