Sabitlenmiş Tweet
Katherine Liu
127 posts
Katherine Liu
@robo_kat
Senior Research Scientist @ToyotaResearch, previously Robotics PhD @MIT_CSAIL. Towards generalist embodied intelligence 🤖 Opinions my own!
Katılım Nisan 2018
210 Takip Edilen188 Takipçiler
Katherine Liu retweetledi
Releasing RecGen: a collaboration between @ToyotaResearch, @toyota_europe, and @UvA_Amsterdam tackling a core 3D vision challenge: reconstructing complete multi-object scenes (parts, poses, textures, even occluded geometry) from just 1 to a few RGB-D views.
Trained purely on synthetic data, RecGen achieves SOTA on real-world robotics and 6D pose benchmarks, handling occlusions, symmetry, and complex interactions.
A step toward scalable, high-fidelity digital twins for robotics, and better evaluation and training of generalist policies.
reconstruction-by-generation.github.io
English
Katherine Liu retweetledi
Katherine Liu retweetledi
Gave a talk on Robot Learning from Human Data at Stanford. It was great to be back!
Some opinionated points:
1. Human data collection capacity is outpacing the research.
2. We still don't have the "science" for scaling robot capability with human data.
3. We are far from being able to model naturalistic human behaviors.
youtube.com/watch?v=NUtaN1…
YouTube
English
Katherine Liu retweetledi
[5/5] But wait, there’s more! Bring your own datasets, load from LeRobot, visualize everything, and explore all tutorials 🧩: github.com/TRI-ML/vla_fou…
Project website: tri-ml.github.io/vla_foundry/
English
Katherine Liu retweetledi
[4/5] Grand finale: evaluate your VLA Foundry robot policies in simulation on 40+ challenging manipulation tasks! 🤖: github.com/TRI-ML/vla_fou…
English
Katherine Liu retweetledi
[3/5] Load our released LLMs and VLMs from HuggingFace, and do inference on your own data ⚡(text completion & image captioning): github.com/TRI-ML/vla_fou…
English
Katherine Liu retweetledi
[2/5] Train the entire LLM → VLM → VLA pipeline, all on one place🧠: github.com/TRI-ML/vla_fou…
English
Katherine Liu retweetledi
[1/5] Want to build your own LLM → VLM → VLA models without the usual headache? VLA Foundry makes it *super* easy. 🛠️ Get started with self-contained tutorial notebooks that anyone can run: tri-ml.github.io/vla_foundry/do…. Examples in 🧵
English
A few interesting rollouts from the Foundry-QwenVLA-2.5B multi-task model on seen tasks in sim – a 🧵. I really like behaviors that involve non-prehensile manipulation, like the little nudges in StoreCerealBoxUnderShelf.
Jean Mercat@MercatJean
Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.
English
Katherine Liu retweetledi
In LBM Eval, our Drake-based multi-task tabletop manipulation simulation:
• Our from-scratch VLA matches our prior LBM
• Our VLA using a Qwen3-VL backbone tops that by over +20%
English
Katherine Liu retweetledi
VLA Foundry supports both from-scratch training and pretrained HF backbones. We trained two model types to show it off:
• Fully from scratch via our LLM→VLM→VLA pipeline
• Built on Qwen3-VL
We share the weightshttps://huggingface.co/collections/TRI-ML/vla-foundry
English
Also, if you’re wondering how we generated all these cool videos from the Drake sim, check out @NicholasEPfaff’s repo github.com/nepfaff/drake-… as a starting point 👀
Katherine Liu@robo_kat
A few interesting rollouts from the Foundry-QwenVLA-2.5B multi-task model on seen tasks in sim – a 🧵. I really like behaviors that involve non-prehensile manipulation, like the little nudges in StoreCerealBoxUnderShelf.
English
Katherine Liu retweetledi
Having control over upstream LLM/VLM training is key to training a good robotics model.
We hope VLA Foundry opens the door for researchers and practitioners to answer questions they previously wouldn’t even have thought of asking if upstream pretraining was simply inherited!
Jean Mercat@MercatJean
Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.
English
You can check out more rollouts on our website tri-ml.github.io/vla_foundry, or use our checkpoints huggingface.co/collections/TR… to get a feel for model performance from our codebase github.com/TRI-ML/vla_fou…
English
And here’s another (PushCoasterToMug). Pushing requires a different (arguably harder) type of physical understanding than simple pick and place and precision really matters here. This is the task with the lowest success rate for this model.
English