Caelan Garrett

Check out Yash Narang's GTC talk today where he will highlight some of our work on GPU-accelerated multi-arm manipulation planning! nvidia.com/gtc/session-ca… github.com/NVlabs/Schedul…

Can large-scale sim data enable real-world generalization?🤔 In our new work, we introduce a generalizable domain adaptation setting, where policies must handle real-world situations never presented in the real training data. (1/n)

Check out this NVIDIA blog about my research on modern robot manipulation planning: developer.nvidia.com/blog/r2d2-perc…

My team at NVIDIA Research in Seattle is looking for senior PhD robotics research interns for summer 2026. Apply online at nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAEx…, and mention the Seattle Robotics Lab (SRL) in your application!

NVIDIA's Seattle Robotics Lab (SRL) is currently hiring PhD and senior (post-PhD) full-time robotics research scientists. Apply online at: nvidia.wd5.myworkdayjobs.com/NVIDIAExternal… nvidia.wd5.myworkdayjobs.com/NVIDIAExternal…

Check out this MIT News article about our research on GPU-accelerated manipulation planning! news.mit.edu/2025/new-syste… @WillShenSaysHi @nishanthkumar23 @imankitgoyal

Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. skillgen.github.io 1/

Want your robot to make a cup of coffee but don’t want to spend hours collecting demos? We introduce SPIRE, a system that solves long-horizon manipulation tasks with limited demos through planning, Behavior Cloning, and Reinforcement Learning. #CoRL2024 #NVIDIAResearch 👇 🧵 1/

Our internship positions are open: nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAEx… Consider applying if you are doing research in foundational model for robotic manipulation such as pick and place, bimanual/dexterous manipulation, tactile and assembly. Feel free to reach out as well after you apply.

Can we teach a robot hundreds of tasks with only dozens of demos? Introducing NOD-TAMP: A framework that chains together manipulation skills from as few as one demo per skill to compositionally generalize across long-horizon tasks with unseen objects and scenes. (1/N)

Tired of collecting interventions all day to train with DAgger? Introducing IntervenGen from @NVIDIARobotics + @berkeley_ai. From just 10 corrective human interventions, IntervenGen generates 1000+ to cover broad robot mistake distributions 👇 arxiv.org/abs/2405.01472 🧵 1/

Great to collaborate with @AjayMandlekar @CaelanGarrett @Ken_Goldberg and Dieter Fox! Paper: arxiv.org/abs/2405.01472 Website: sites.google.com/view/interveng…

We have research internship positions in our team for the Summer of 2024. If you are passionate (and experienced) in using learning, LLM/VLMs, and 3D reasoning to take the robot manipulation capabilities to the next level, apply to our team: nvidia.wd5.myworkdayjobs.com/NVIDIAExternal…

This project was partially conducted while I was an intern at NVIDIA Research. Big thanks to @CaelanGarrett and Dieter Fox from NVIDIA Seattle Robotics Lab, as well as my advisors Leslie Kaelbling and Tomás Lozano-Pérez from @MIT_LISLab for this fruitful collaboration! 😆🩷🤖 9/N

Imitation learning is powerful, but hard to scale due to lack of high quality data. Introducing Optimus: TAMP-supervised visuomotor transformers. Optimus solves over 300 long-horizon manipulation tasks with up to 8 stages and 72 different objects. @NVIDIAAIDev @CMU_Robotics 1/N

If you're attending #ICRA2022, check out recent work led by @AidanCurtis3, @xiao_lin_fang and @CaelanGarrett on enabling task and motion planning to manipulate unknown objects (with a real robot!). Talk will be at 10:45 AM 05/24 in Room 112B Paper: arxiv.org/abs/2108.04145

Classical AI planning algorithms are quite general-purpose, but are typically limited to finite action spaces and thus cannot model continuous robotic systems. At ICAPS 2020, we presented a new planning framework that overcomes this limitation. Paper: arxiv.org/abs/1802.08705

Planning for multi-object robot manipulation has been studied in several different communities; our new survey paper works toward a unifying perspective. Paper: "Integrated Task and Motion Planning" (arxiv.org/abs/2010.01083) TAMP example: youtu.be/VZmfC_RWlps

Hello, world! We are the Learning and Intelligent Systems group @MIT_CSAIL, headed by Leslie Pack Kaelbling & Tomás Lozano-Pérez. We work on AI, ML, and robotics, and we’ll be mostly tweeting about new work by our group.

@aadubredu Thanks for the shoutout! I’m glad you enjoyed it :)

Brilliant talk by Caelan Garrett (@CaelanGarrett) on his work on PDDLStream, learning samplers for planning and on planning in belief space for partially observed domains: youtu.be/YfCh5qMYGuQ