Marco Pavone

Verification is emerging as a new scaling axis for AI! Scaling pre-training, post-training, and test-time compute have driven much of the recent progress in large language models. Our new work explores a fourth scaling axis: #verification —the ability to determine whether a solution is actually correct. In LLM-as-a-Verifier, we introduce a general-purpose framework that provides fine-grained feedback across diverse modalities without additional training. We show that three simple ingredients—higher score granularity, repeated evaluation, and criteria decomposition—consistently improve verification performance. The approach achieves state-of-the-art results across robotics, coding, and medical AI, including RoboRewardBench, Terminal-Bench V2, SWE-Bench Verified, and MedAgentBench. I'm particularly optimistic about the implications for #Robotics and #PhysicalAI, where verification can serve as a dense reward signal for reinforcement learning, significantly improving the sample efficiency of SAC and GRPO and, in turn, enabling more capable and reliable autonomous systems. As AI continues to scale, I believe verification will become a foundational capability for building more capable and trustworthy autonomous AI agents. 🌐 Website: llm-as-a-verifier.com 📄 Paper: arxiv.org/abs/2607.05391 💻 Code: github.com/llm-as-a-verif… Outstanding work led by @jackyk02, in collaboration with @shululi256, @pranav_atreya, @liu_yuejiang, @jyx_su, @chelseabfinn, @istoica05, and @Azaliamirh. #AI #LLMs #Verification #Reasoning #AgenticAI #Robotics #PhysicalAI #ReinforcementLearning @StanfordAILab @StanfordEng

How do we make robot policies robust to rare but high-impact failures? Video #World #Models (WMs) are rapidly becoming a powerful tool for robotics, enabling policy evaluation and improvement by "imagining" future outcomes. But there's a catch: these imagined futures are typically nominal samples, making it easy to overlook the rare yet safety-critical events that matter most. In our new paper, StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement, we explore a simple but powerful idea: 💡 Instead of passively sampling futures, actively steer world model imaginations toward high-impact yet still plausible scenarios. StressDream optimizes the initial diffusion noise at inference time, allowing us to generate targeted stress-test scenarios without retraining the world model. This enables: - More robust policy evaluation by exposing failure modes that random sampling often misses. - Improved policy optimization by training against challenging but realistic imagined futures. As generative world models become a foundation for #Physical #AI, the ability to systematically probe their "long tail" of plausible futures will be increasingly important for building reliable and trustworthy autonomous systems. 📌 𝖯𝗋𝗈𝗃𝖾𝖼𝗍 𝖯𝖺𝗀𝖾: junwon.me/StressDream/ 📄 𝖯𝖺𝗉𝖾𝗋: arxiv.org/abs/2606.00267 Work led by Junwon Seo, with a great set of collaborators: Sushant Veer, Thomas Ran Tian, Wenhao Ding, Apoorva Sharma, Karen Leung, Edward Schmerling, Andrea Bajcsy. @NVIDIADRIVE @NVIDIAAI #Robotics #WorldModels #PhysicalAISafety #AISafety #AutonomousSystems #RobotLearnin

Introducing ConstrainedMimic (arxiv.org/pdf/2606.00374) — a control framework for #humanoid robot #safety that enables real-time constraint enforcement within #RL-based motion tracking policies by leveraging whole-body kinematics and dynamics. Recent advances in reinforcement learning have unlocked remarkable whole-body agility for humanoid robots. However, ensuring safety and satisfying constraints—especially those introduced after training—remains a significant challenge for deploying safe and reliable systems. ConstrainedMimic addresses this challenge by combining ideas from operational space control and control barrier functions (CBFs). The framework enables enforcement of arbitrary runtime constraints while preserving the ability of the policy to track complex motions. Importantly, constraints can be imposed on both the kinematic reference motion and the underlying robot dynamics, providing a principled approach to safer, more robust, and more controllable humanoid behavior. As #PhysicalAI, #humanoid #robotics, and #embodied #AI systems move from research environments into the real world, the ability to guarantee safety and respect operational constraints will become increasingly important — ConstrainedMimic is a step in this direction. 📄 Paper: arxiv.org/pdf/2606.00374 💻 Code: Coming soon Great work led by @danielpmorton . #PhysicalAI #AISafety #HumanoidRobotics #EmbodiedAI #ReinforcementLearning #Robotics @StanfordAILab @StanfordEng

Today we're releasing @nvidia AlpaGym, our new open-source reinforcement learning (RL) framework for end-to-end autonomous driving. A key challenge for #Physical #AI is enabling policies to learn from the consequences of their actions. While supervised learning can teach a model to imitate behavior, robust autonomy ultimately requires learning through interaction with the environment. AlpaGym enables exactly that. Built on top of: - AlpaSim: our high-fidelity closed-loop autonomous driving simulator - Cosmos-RL: NVIDIA's distributed RL training and rollout infrastructure AlpaGym provides the glue that connects simulation, training, and driving policies into a scalable framework for post-training autonomous vehicle models in closed loop. With AlpaGym, researchers and developers can: ✅ Train end-to-end driving policies using reinforcement learning ✅ Run large-scale closed-loop simulations ✅ Experiment with new reward functions, policy architectures, and training strategies ✅ Benchmark models on public leaderboards 📖 Learn how it works: developer.nvidia.com/blog/how-to-po… 💻 GitHub: github.com/NVlabs/alpagym 🏆 Open Challenges: - AlpaSim Closed-Loop E2E Driving Challenge: huggingface.co/spaces/nvidia/… - Physical AI AV Reasoning Challenge: huggingface.co/spaces/nvidia/… Learn more about the #Alpamayo open platform: huggingface.co/blog/drmapavon… #PhysicalAI #AutonomousDriving #ReinforcementLearning #Robotics #OpenSource #NVIDIA #MachineLearning @NVIDIADRIVE @NVIDIAAI

On the power of data flywheels in Physical AI @nvidia recently introduced Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, images, video, audio, and actions within a unified mixture-of-transformers architecture. One aspect I find particularly exciting is the data flywheel emerging between #Cosmos 3 and #Alpamayo 2. Cosmos 3 was trained using data generated and curated in part through the Alpamayo ecosystem. In turn, the next generation of Alpamayo will build on Cosmos 3's capabilities. This creates a virtuous cycle in which better models enable better data generation, and better data leads to even stronger models. Much of the attention in #AI is naturally focused on model architectures and benchmark results. Yet, in robotics and autonomous systems, development processes matter just as much as models themselves. Robust data flywheels are increasingly becoming a defining characteristic of state-of-the-art robot autonomy stacks. Further reading: - Cosmos 3: arxiv.org/pdf/2606.02800 - Alpamayo 2: huggingface.co/blog/drmapavon… Join me on June 16 at 9:00 AM PT for a livestream on Alpamayo 2 Super: The Open Reasoning Model for Robotaxis: addevent.com/event/5djffx3c…. We'll showcase brand-new elements of the open pipeline—from real-world fleet data to model training recipes to closed-loop development with simulation. If you're building toward L4 autonomy, I think you'll find it worthwhile. I'll be joined by @iamborisi @YurongYou @yan_wang_9 @MaxiIgl Looking forward to seeing you there. @NVIDIADRIVE @NVIDIAAI

How much time should robots spend thinking? Vision-Language Models are increasingly used as high-level planners for robots, and the prevailing strategy has been to scale test-time compute to boost capability. But more reasoning steps, bigger models, and longer memory all come with increased latency, tokens, and FLOPs—often with diminishing and uneven returns. So when, and where, is test-time compute actually worth its cost? 🧐 We study three dominant scaling axes and find that each unlocks a distinct capability, showing that test-time compute is not a uniform lever: - Chain-of-thought depth helps with tasks involving implicit semantic, physical, or spatial constraints, but its additional latency is not always necessary (on VLABench, a non-CoT model matches a CoT model on 44% of tasks). - Model size governs the breadth of skills a planner can reliably draw upon, but its benefits appear only when those additional skills are actually required. - Memory history improves performance on long-horizon, history-dependent tasks, but can actively hurt performance elsewhere. Across all three axes, a consistent pattern emerges: the gap between cheap and expensive configurations is large, but highly non-uniform and task-dependent. DIRECT (Dynamic Inference Router for Embodied Compute Tradeoffs) is a lightweight router that reads scene + instruction context and sends each task to the cheapest planner that can still solve it, allocating compute per task rather than committing to one fixed model. 👉 Takeaway: smart allocation of test-time compute can recover frontier-level planning at a fraction of the cost. 📄 Paper: arxiv.org/abs/2606.12402 🔗 Website: jadee-dao.github.io/direct/ Work led by @_jadelynn @milanganai With an outstanding team of collaborators: @ajaysridhar0 @Mozhgan_nasr @katielulula Clark Barrett @jiajunwu_cs @chelseabfinn #Robotics #VLM #EmbodiedAI #MachineLearning #TestTimeCompute

The Alpamayo Summit at CVPR brought together AV researchers and industry leaders together under one room. Hear from Marco Pavone (@drmapavone), Senior Director of Autonomous Vehicle Research, and other NVIDIA experts on how Alpamayo is accelerating AV development. 📺 Watch the on-demand replay: nvidia.com/en-us/on-deman…

I look forward to participating in the Verification Summit (verificationsummit.ai) and sharing my perspective on Physical AI safety. I strongly agree that verification and validation are key frontiers for unlocking Physical AI in high-stakes, high-reliability applications, from autonomous cars to industrial robotics! @fv_summit @khoslaventures @PramaanaLabs @boldcapfund

Excited to share the latest expansion of the @nvidia #Alpamayo open platform for reasoning-based autonomous vehicles. Since its launch earlier this year, Alpamayo has seen rapid adoption across industry and academia, with its reasoning models surpassing 400,000 downloads and earning a #COMPUTEX 2026 Best Choice Award. As announced by Jensen Huang during his #COMPUTEX keynote, we are now introducing several major additions designed to accelerate the development of next-generation AV systems (more details here: huggingface.co/blog/drmapavon…): 🚗 Alpamayo 2 Super — a new 32B-parameter driving foundation model with: • Full 360° surround-view perception • Advanced reasoning capabilities and chain-of-causation outputs • Meta-actions such as lane changes, yielding, and stopping • Reasoning auto-labeling and visual grounding for scalable data annotation • State-of-the-art performance across reasoning, prediction, and alignment tasks 🔄 AlpaGym — an open-source framework for closed-loop reinforcement learning, enabling AV models to learn from the consequences of their actions in simulation and helping bridge the gap between training and real-world deployment. 📊 New Open Benchmarks — including challenges for closed-loop driving and long-tail reasoning to help the community measure progress and drive innovation. 🛠️ Alpamayo Recipes — a centralized repository of end-to-end workflows covering supervised fine-tuning, reinforcement learning, quantization, and model customization. Reasoning models and closed-loop training are becoming foundational technologies for autonomous systems. Our goal is to provide the open tools, models, infrastructure, and benchmarks needed to accelerate progress across the entire AV ecosystem. A huge thank you to the many researchers, engineers, and community members whose feedback helped shape this release. Resources: • Overview of the latest Alpamayo release (note: some components will be released over the coming weeks): huggingface.co/blog/drmapavon… • @nvidia announcement: nvidianews.nvidia.com/news/nvidia-al… #AutonomousVehicles #PhysicalAI #Robotics #AI #MachineLearning #ReinforcementLearning #OpenSource #NVIDIA #Alpamayo @NVIDIADRIVE @NVIDIAAI

We’ve just released the #Alpamayo Chain-of-Causation (CoC) Autolabeling Pipeline — a feature that has been highly requested by the community! The pipeline automatically derives: 🔹 Meta-actions: high-level categorical descriptions of ego motion 🔹 Chain-of-causation labels: causal links between scene factors and the ego vehicle’s intended behavior Autolabeling pipeline: github.com/NVlabs/alpamay… Learn more about the Alpamayo open platform: huggingface.co/blog/drmapavon… We’re excited to see what the community builds with it, and we hope this tool will help accelerate research in the rapidly growing area of #reasoning models for #Physical #AI. @NVIDIADRIVE @NVIDIAAI

Thrilled to personally invite you to the @nvidia #Alpamayo Summit at #CVPR! I’ll be opening the event with a talk titled “The ChatGPT Moment for Autonomous Driving” — exploring how reasoning AI is reshaping the entire autonomy stack and accelerating the path toward scalable, safe Level 4 autonomous driving. 📍 June 4, 2026 📍 Le Méridien Denver Downtown (Grove Ballroom) 🕞 3:30 PM — Networking + Snacks 🕓 4:00 PM — Program Begins We’ll cover: • Open datasets • New reasoning models • AlpaSim • Safety frameworks • And much more Join the waitlist here → nvevents.nvidia.com/alpamayo-summit The event is currently sold out, but you can still join the waitlist. @NVIDIADRIVE @NVIDIAAI

Autonomous vehicle technology is advancing at an unprecedented pace. Marco Pavone (@drmapavone), Senior Director of Autonomous Vehicle Research at NVIDIA, breaks down how AI is enabling developers to completely rethink how autonomous systems are built. 📺 Watch the full video: nvda.ws/43rYlJy

🚗🏆 Breaking news: @nvidia #Alpamayo open platform has been named the winner of a COMPUTEX TAIPEI 2026 Best Choice Award in the Vehicle Technology & Smart Cockpit category, recognizing Alpamayo as one of the year’s major breakthroughs in automotive and Physical AI technology! Announcements: - #bca-awards" target="_blank" rel="nofollow noopener">blogs.nvidia.com/blog/nvidia-gt… - bcaward.computex.biz/WinnerYearDeta… I’m incredibly proud of the team behind the Alpamayo open platform — Wenjie Luo @yan_wang_9 @iamborisi and the entire NVIDIA Autonomous Vehicle Research Group — and deeply grateful for the contributions from the NVIDIA AV production team Xinzhou Wu Sarah Tariq and many other researchers and developers across NVIDIA. This achievement was truly a collective team effort.👏 To get started with Alpamayo: — huggingface.co/blog/drmapavon… — huggingface.co/blog/drmapavon… And stay tuned — we’ll have several exciting announcements in the coming weeks. @NVIDIADRIVE @NVIDIAAI

Introducing FRAX: Fast Robot Kinematics and Dynamics in #JAX — to be presented at the 2026 IEEE International Conference on Robotics and Automation (ICRA) Frontiers of Optimization for Robotics (FOR) Workshop. FRAX delivers extremely fast (low-microsecond) execution for common inverse-kinematic and inverse-dynamic control workloads, with a pure Python codebase that can achieve up to 5× faster performance than MuJoCo or Pinocchio Python bindings in several settings. At the same time, FRAX is fully differentiable and seamlessly compatible with CPU, GPU, and TPU execution through #JAX — enabling scalable workflows spanning robotics, control, planning, and machine learning. Our broader goal is to help bridge the gap between modern AI tooling and robotics computation, making it easier to develop scalable #Physical #AI systems. This also makes FRAX a great complement to CBFPY (github.com/StanfordASL/cb…), our package for robot safety and control barrier functions. Kudos to @danielpmorton for leading this effort. If you’ll be at ICRA, reach out! The FOR Workshop is on Monday, June 1, and we’ll have a poster there. 💻 GitHub: github.com/StanfordASL/fr… 📄 Paper: arxiv.org/pdf/2604.04310 #Robotics #PhysicalAI #JAX #DifferentiablePhysics #MachineLearning #AutonomousSystems #GPU #Simulation #ICRA

Excited to announce the launch of the Stanford Sustainable Mobility Center, where I’ll be serving as inaugural co-director. Housed within Stanford Precourt Institute for Energy, the center brings together @Stanford’s strengths — from energy systems to AI and autonomy — alongside industry and government collaboration to accelerate real-world mobility solutions at scale. 🔗 Overview of the center: news.stanford.edu/stories/2026/0… The center traces its origins to the Center for Automotive Research at Stanford (CARS), which I had the pleasure of directing for several years. 🚗🚢✈️ If you are interested in rethinking how people and goods move across land, sea, and air, I’d love to connect! @StanfordEng @StanfordASL

A central challenge in #physical #AI is data scarcity: vision-language-action (#VLA) models are fundamentally limited by the availability of high-quality robotics demonstrations. In our recent work, we introduce R&B-EnCoRe (arxiv.org/pdf/2602.08167), a framework that enables models to self-bootstrap embodied #reasoning by leveraging synthetic visuo-textual data together with limited embodiment-specific experience. In essence, R&B-EnCoRe allows models to learn how to reason in an embodied setting. Our approach treats reasoning as a latent variable and uses self-supervised refinement to learn reasoning strategies that are directly predictive of successful control—without human annotations, reward engineering, or external verifiers. We validate the approach across a range of embodiments—including manipulation, navigation, and autonomous driving—and across model scales from 1B to 30B parameters, observing consistent improvements: 💪 +28% task success in real-world manipulation 🦿 +101% score in legged locomotion navigation 🚗 −21% collision rate in autonomous driving Overall, this work highlights a promising direction: aligning internet-scale priors with embodiment-specific data to enable scalable, self-improving physical intelligence. Kudos to an amazing team: Milan Ganai Katie Luo @JonasFrey96 Clark Barrett 🌐 Website: milanganai.github.io/rnb-encore/ 📄 Paper: arxiv.org/pdf/2602.08167

Excitingly, @nvidia #Alpamayo 1.5 is now available within Autoware: github.com/autowarefounda… Grateful to @ShinpeiKato and the rest of the TIER IV team for helping democratize the development of AV solutions. I look forward to seeing #Alpamayo’s adoption continue to grow! As Jensen said, “Everything that moves will be autonomous.” Together, we are making big strides toward this vision! More about Alpamayo 1.5: huggingface.co/blog/drmapavon… @NVIDIADRIVE @NVIDIAAI