Fang Jiading

Just realized they indeed released a post about robotics a several days ago: anthropic.com/research/claud…

So, Anthropic is building self-replicating robots right now? 😂

[5/5] Camera conditioning improves performance across ACT, Diffusion Policy, and SmolVLA, in both simulation and the real world. Key takeaway: Robot policies may spend substantial capacity inferring camera pose from appearance cues. When camera pose is available (or can be estimated), providing it explicitly can significantly improve viewpoint generalization. Full video below 👇 Paper: arxiv.org/abs/2510.02268 Project: ripl.github.io/know_your_came… code: github.com/ripl/CamPoseOp… Congratulations to lead author @JiangTianchong and the entire team: Jingtian Ji, Xiangshan Tan, @jiading_fang, @vitorguizilini, @mattrwalter

Really glad to see another MLLM conditioned by cameras! Check our work conditioning robot learning models on cameras (recently received best robot learning finalist award in ICRA 2026): ripl.github.io/know_your_came… Conditioning on 3D signals is not against end-to-end approaches, instead it exposes the importance of physics grounding in MLLMs, especially when they are relatively cheap and reliable to obtain. I hope it inspires the development of next-gen models that truly understands the physical world.

Glad this shell game task gets more attention from @GeneralistAI and @RhodaAI for measuring long-context robot manipulation capabilities. We explored this robotic task 3 years ago with state-based LLMs. Check out Statler: arxiv.org/abs/2306.17840.

This is a quite important question because it’s about how to enforce HARD constraints, the very problem people facing trying to use world models for physics simulation. Many comments mention data + RL. I get the data part; and for the RL part, it’s possible to do OCR or finger detection, but how to RL physics?

For Vincent to come out and cannibalize his very own contributions to the field—3D and scene reps—is an absolutely courageous and chivalrous move. It makes this piece painfully credible. Real-world intelligence is the only lead.

We’re excited to introduce the Waymo World Model—a frontier generative mode for large-scale, hyper-realistic autonomous driving simulation built on @GoogleDeepMind’s Genie 3. By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. waymo.com/blog/2026/02/t…

Presenting tomorrow (10/21 Tuesday) afternoon at poster session 2 #359. Let’s discuss about reconstructing articulated realistic environments for robot simulations🤖! #ICCV2025

Great work Ankit! Feels the bitter-lesson all-over again smh. Retaining the original vocabulary is probably a big part, although it may be hard to properly ablate. I used to not believe that VLMs can do geometric or numerical works better than vision/action-specific models, but more and more evidences are proving me wrong.

What's the right architecture for a VLA? VLM + custom action heads (π₀)? VLM with special discrete action tokens (OpenVLA)? Custom design on top of the VLM (OpenVLA-OFT)? Or... VLM with ZERO modifications? Just predict action as text. The results will surprise you. VLA-0: Outperforms π₀, GR00T-N1, MolmoAct, SmolVLA. With ZERO changes to the VLM. 🧵⬇️

This is the last piece of work from my PhD (from almost one year ago). And I'm proud of the solid study we have done. Huge shoutout to the project lead @JiangTianchong , and colleagues Jingtian Ji, Xiangshan Tan to make my wild thought a reality. They did excellent work in implementation and uncovered many crucial tricks to enable the model. Also huge thanks to all the collaborators @anand_bhattad, Vitor Guizilini and Matthew Walter for continuous support and guidance along the way. Please let us know if you have any questions, this is only a first step towards answering a grand quest.

The idea originated from the observation that, at the time, almost none of the imitation learning or VLA models take into input the camera parameters like extrinsics and intrinsics, but just using RGB frames. Often the times, people don't even bother to properly calibrate the camera, hoping that large-scale training can solve it all. While such dream is not impossible from a scaling perspective, one has to consider the data scarcity and efficiency in reality. From the 3D community, we know it's difficult to reconstruct the scene if the camera parameters are off. If we assume 3D understanding is the pre-requisite of learning robot motion, the current approaches for VLAs are basically trying to solve a harder task with even less data. It creates a big gap in our understanding. Thus we set up to study the element of camera inside VLA or imitation learning models.

🤖Ever wonder why robotic IL models/VLAs never make use of the camera info (extrinsics & intrinsics)? We wonder the same. So we created ways to inject camera info and carefully studied its impacts. In fact, it's important to "Know Where Your Camera Is"! arxiv.org/abs/2510.02268

"the human brain doesn't need tons of training data to do stuff"

@ssahoo_ “Romantic” is one of the best descriptions for a PhD life.

For a PhD, you need to be a romantic at some level. Your papers will get rejected. Your ideas will get scooped. All while you peers flourish. And yes--It will sting. 2023 was one such year for me. Yet I call it my golden year, because that’s when I truly fell in love with my research. I found my escape in it.

Interesting paper proving that fixed-size single-vector embedding is not enough for retrieval tasks because the dimension required can be arbitrarily high (up to the size of the system itself where the vector provides no compression). Its implications go beyond retrieval. While the paper mainly considers the retrieval tasks, the results apply to other domains using similar techniques. On top of my head, visual-text embedding like CLIP or CoCA; VQ-VAE methods. The usages of CLIP or CoCA are mainly a) for contrastive learning between language and image embeddings, such as zero-shot classification. The result directly applies, meaning no dimension is high enough, and there are going to be images that are misclassified. b) for image embedding that combines with language tokens to be decoded by language decoders. This is similar to the "cross-encode" exception where the query text and the retrieval are encoded together and can achieve 100% accuracy on the LIMIT data using Gemini-2.5-pro. VQ-VAE is another popular "compression" method used for image reconstruction and generation, where the representation is a set of vectors from a fixed dictionary. In this case, the representation belongs to the "multi-vector" category, and is not cursed by the results.