RoboPapers

Achieving generalizable manipulation is the north star for robotics learning, and while we’ve in the past seen incredible results on specific tasks using fine-tuned VLAs, this north star has remained elusive. Perhaps what is needed is a different approach. DreamZero proposes World Action models (WAMs), which jointly model both action and video in order to achieve state-of-the-art performance on benchmarks like MolmoSpaces and RoboArena. @SeonghyeonYe of @NVIDIARobotics joins us to talk about building a 14B parameter autoregressive diffusion model which achieves state-of-the-art generalization on real world tasks and on the best available benchmarks. Watch episode #68 of RoboPapers, with @micoolcho and @chris_j_paxton, now!

Full episode dropping soon! Geeking out with @SeonghyeonYe on DreamZero: World Action Models Are Zero-Shot Policies dreamzero0.github.io @NVIDIARobotics Co-hosted by @micoolcho @chris_j_paxton

Robotics research is moving fast, and being able to modify and improve upon hardware is crucial to maintaining velocity. That’s why Menlo Research has started working on their own open-source humanoid project, Asimov @asimovinc. And they are moving fast. It’s been roughly six months since they started the project, and they already have full humanoid with arms, legs, and a head, which can walk forwards and backwards. @selim__1903 and @AlexE_00 of Menlo Research join us to talk about the development of this open-source humanoid. Watch episode 67 of RoboPapers, with @chris_j_paxton and @DJiafei, now! And follow @asimovinc for daily updates on humanoid robot development.

Full episode dropping soon! Geeking out with @SeonghyeonYe on DreamZero: World Action Models Are Zero-Shot Policies dreamzero0.github.io @NVIDIARobotics Co-hosted by @micoolcho @chris_j_paxton

Full episode dropping soon! Geeking out with @selim___1903 @AlexE_00 on Asimov github.com/asimovinc/asim… Co-hosted by @chris_j_paxton @DJiafei

Full episode dropping soon! Geeking out with @selim___1903 @AlexE_00 on Asimov github.com/asimovinc/asim… Co-hosted by @chris_j_paxton @DJiafei

How should we represent robot actions for autoregressive transformers? Most robot policies use diffusion or flow to generate continuous action sequences, but this isn’t how large language models work; they predict output tokens, which has many advantages. But coming up with a set of useful action tokens, so we can skip the slow and expensive diffusion steps, is difficult. @liu730chaoqi says action tokens need three qualities: reasonable compression, universal decodability, and a left-to-right causally ordered token space, and he proposes Ordered Action Tokenization as a solution to all three. Watch Episode 66 of RoboPapers now, with @micoolcho and @chris_j_paxton, to learn more!

Full episode dropping soon! Geeking out with @liu730chaoqi on OAT: Ordered Action Tokenization ordered-action-tokenization.github.io Co-hosted by @micoolcho @chris_j_paxton

Full episode dropping soon! Geeking out with @liu730chaoqi on OAT: Ordered Action Tokenization ordered-action-tokenization.github.io Co-hosted by @micoolcho @chris_j_paxton

Learn more: Project page: cladernyjorn.github.io/VLM4VLA.github… ArXiV: arxiv.org/abs/2601.03309 Code: github.com/CladernyJorn/V…

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance? Jianke Zhang and @GYanjiang join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary tasks like quesiton answering did not lead to downstream improvements in control. To learn more, watch episode 65 of RoboPapers now, with @chris_j_paxton and @DJiafei!

Full episode dropping soon! Geeking out with Jianke Zhang @GYanjiang on VLM4VLA: Revisiting Vision-Language Models in Vision-Language-Action Models cladernyjorn.github.io/VLM4VLA.github… Co-hosted by @chris_j_paxton @DJiafei

Full episode dropping soon! Geeking out with Jianke Zhang @GYanjiang on VLM4VLA: Revisiting Vision-Language Models in Vision-Language-Action Models cladernyjorn.github.io/VLM4VLA.github… Co-hosted by @chris_j_paxton @DJiafei

Human motion is instinctual. We know how to interact with the world around us, almost without thinking about it at all. @ziwenzhuang_leo and @ShaotingZ38103 joined us on RoboPapers to talk about their ambitious Project Instinct: which provides the tools, algorithms, and environments necessary to build humanoid whole-body control which can handle contact with the environment. Watch Episode #64 of RoboPapers with @micoolcho and @DJiafei now!

Full episode dropping soon! Geeking out with @ziwenzhuang_leo @ShaotingZ38103 on Project Instinct project-instinct.github.io Co-hosted by @micoolcho @DJiafei

Full episode dropping soon! Geeking out with @ziwenzhuang_leo @ShaotingZ38103 on Project Instinct project-instinct.github.io Co-hosted by @micoolcho @DJiafei

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by @Hongyu_Lii and @jiahuifu_carol from RAI. Watch Episode #63 of RoboPapers with @chris_j_paxton and @micoolcho now to learn more!

Full episode dropping soon! Geeking out with @Hongyu_Lii @jiahuifu_carol on NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos novaflow.lhy.xyz Co-hosted by @micoolcho @chris_j_paxton

Full episode dropping soon! Geeking out with @Hongyu_Lii @jiahuifu_carol on NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos novaflow.lhy.xyz Co-hosted by @micoolcho @chris_j_paxton