Pierre Sermanet

@RemiCadene @UMA_Robots @AdilZtn @alibert_s @Mickael_Chen @LeoTronchon @MustafaShukor1 @therobotstudio @marine_caous Lua was a cool language ❤️

@psermanet @UMA_Robots @AdilZtn @alibert_s @Mickael_Chen @LeoTronchon @MustafaShukor1 @therobotstudio @marine_caous I also ported Overfeat to Torch7 in LUA, before pytorch 😅 The repo later became pretrained-models.pytorch: github.com/Cadene/pretrai…

Humanity is at a turning point. I am launching UMA to build general-purpose mobile and humanoid robots from Europe. Proud to start with people I admired for years, and grateful for all your support! Reach out to us @UMA_Robots ❤️

@r0b0t_sp1der @RemiCadene @alibert_s @therobotstudio @Mickael_Chen @ylecun @AdilZtn @LeoTronchon you can apply at uma.bot ! :)

@psermanet @RemiCadene @alibert_s @therobotstudio @Mickael_Chen @ylecun @AdilZtn @LeoTronchon hire me for firmware and embedded and edge models

I’m really happy to share that we’re launching UMA. Together with @RemiCadene, @alibert_s, @therobotstudio, and an exceptional founding team, we’re building general-purpose mobile and humanoid robots. If you want to be part of this adventure, reach out at uma.bot Throughout my career, I have been obsessed with scalable learning and data acquisition methods that require little to no labels. Back in 2005 with @ylecun, we were self-supervising our “deep” 2-layer network to do long range vision using short range stereo information, this was running live onboard our robot. However, because our deep model was so slow, the robot would crash constantly, so I designed a decoupled fast & far architecture for robust navigation, allowing fast control to coexist with slow long horizon thinking, much like systems 1 & 2 in modern humanoids. My PhD was focused on making deep learning work for computer vision, including unsupervised feature learning with @koraykv, writing and open-sourcing a C++ deep learning library with @soumithchintala, and open-sourcing one of the first deep learning vision systems. I came back towards robotics at @Google Brain and @GoogleDeepMind, where I pushed for entirely label-free methods on real robots. In 2017, @coreylynch and I managed to make our robot imitate human motion by co-training self-supervision across sim and real domains jointly, without any labels. With @imkelvinxu and @svlevine , we showed that unsupervised visual reward learning could be used for RL in the real world. In 2020, Corey and I developed the first manipulation VLA, which was trained with very few language labels thanks to self-supervision on play data (playing is an efficient way to demonstrate and practice a broad set of skills and is essential for human development). I was never satisfied with the status quo of top-down data collection, where researchers decide a few tasks to collect data on. Instead, I believed that we should let the data speak: tasks should be automatically discovered bottom-up (scalable and general) from cheap and continuous data collection, with a sprinkle of more expensive data and labels. In 2022, I explored long-horizon reasoning for robotics using scalable automatic labeling augmentations for VQA tasks and studied the economics of different data collection schemes. Most recently, I developed approaches to scalably discover laws of robotics from real data (images, hospital reports, sci-fi literature) in a broad and bottom-up fashion, which improved robot behavior over top-down approaches like Asimov’s laws. All these experiences nourished my vision for UMA as Chief Scientist, I’m incredibly excited to put everything together and so grateful I get to contribute to this incredible moment in human history. Picture: Yann supporting UMA as an advisor and investor, with the team in Paris a couple weeks ago.

@soumithchintala @RemiCadene Thank you so much Soumith for supporting us!

Congrats to @psermanet, @RemiCadene and team on starting UMA - uma.bot . Considering Pierre's track record at Google Robotics, and Remi's phenomenal work on LeRobot at HuggingFace, this is going to be great. I'm so glad to be an angel investor.

@ericjang11 @Google @GoogleDeepMind @V_Vanhoucke Thank you so much Eric this means a lot ❤️, right back at you for the amazing work at 1X!

@psermanet @Google @GoogleDeepMind @V_Vanhoucke congrats! I’ve always been inspired by your taste - the simplicity and the scaling. Very prescient and looking forward to see where you take things

After 11 incredible years at @Google Brain and @GoogleDeepMind, I’m turning the page to start something new in robotics (currently in stealth 👀). I joined Brain in 2014, drawn by the momentum around robotics and machine learning, and I’ll always be grateful to @V_Vanhoucke for creating such a unique space for robotics research and for bringing together some of the brightest minds I’ve ever met. During my time there, I had the joy of exploring ideas that once felt like science fiction: self-supervised imitation from video, unsupervised visual reward models, the first vision-language-action model for manipulation, long-horizon reasoning, and even using sci-fi and Asimov’s laws to improve robot behavior. Huge thanks to my managers Vincent, Anelia, Kanishka, and to all my amazing collaborators: you made this journey unforgettable. After 20 years in robotics, it’s clearer than ever that the field is maturing fast and I feel lucky to keep building through this moment. More soon 🤖

Great to see more robotics projects in Europe🇪🇺

I am starting a venture on top of LeRobot! We’re at a pivotal time. AI is moving beyond digital to the physical world. Embodied AI will change our surroundings in ways we can barely imagine. This technology holds the potential to empower everyone. It must not be controlled by just a few. This conviction led me to propose an ambitious open-source AI robotics project to Thom, Clem, and Julien back in 2024. Hugging Face, home to a community of millions of AI builders and a team of experts who brought us transformers, datasets, and the Hugging Face Hub, was the perfect place to launch LeRobot. I’m incredibly grateful for all the support that allowed me to build LeRobot alongside an amazing team and community. In such a short time, we built one of the most adopted open-source robotics platforms, used by startups, universities, and research labs. It is helping countless people take their first steps in robotics. Together, we’ve even assembled the world’s largest open robotics dataset. And this is only the beginning for LeRobot! Building on this momentum, I now feel the urgency to start something new on top of LeRobot. It will push the limit of what robots are capable of and commoditize them within society. Like LeRobot, it will start in Paris, leveraging its vibrant international AI scene. Stay tuned! As LeRobot continues to expand, it’s now in the best possible hands with @AractingiMichel, @pepijn2233 and Steven Palma taking the lead. Watching the team deliver exceptional results over the last weeks has been one of the most rewarding experiences. Their creativity, dedication, and capability to ship fast is proving just how strong the team is today! I am extremely grateful to the many people who contributed to making LeRobot at Hugging Face and within its powerful community. Many thanks to Thom, Clem, Julien, Simon, Rob, Michel, Pepijn, Steven, Gloria, Adil, Martino, Caroline, Marine, Mishig, Guillaume, Pablo, Lysandre, Arthur, Quentin, Florent, Brigitte, Victor, Marina, Mustafa, Francesco, Jess, Jade, Ville, Leo, Max, Julien, Alexander, Flavien, Raphael, Adina, Tao, Dana, Batu, Olivier, Matthieu, Eugene, Theo, Guilherme, Hynek, Loubna, Clémentine, Merve, Vaibhav, Anna, Jeff, Adrien, Emily, Johanne, Adrien and others. There are too many of you to be all named! Thanks again and see you soon!!! :) ~ Remi

@RemiCadene Super exciting!

While the findings are positive, it does not mean that AI cannot be manipulated for negative outcomes. By releasing this benchmark and dataset, we hope to further safety research and ethical deployment of AIs and robots. Authors: @psermanet, @Majumdar_Ani, @vikassindhwani

We are grateful to Sci-Fi authors for the potential positive real-world impact as we find that Sci-Fi-inspired robot constitutions yield some of the most aligned behavior when measured against realistic scenarios in the ASIMOV benchmark asimov-benchmark.github.io

So how would AI-powered robots behave if we dropped them into Science Fiction literature? And can we generate useful robot constitutions 📜 from Sci-Fi? We generate the first Sci-Fi-inspired constitutions and large-scale ethics benchmark scifi-benchmark.github.io @GoogleDeepMind

We dedicate this work to Isaac Asimov who continues to be a source of inspiration. Authors: @psermanet, @Majumdar_Ani, @AlexIrpan, D. Kalashnikov, @vikassindhwani

While the generated constitutions are for research purposes only and not intended for deployment, we hope that this release can help increase the safety of all robots in the future as much research remains needed to make robots safer before deployment.

Q: How can we ensure robots behave properly at scale? A: Robot constitutions 📜! Q: How do we verify behavior in undesirable situations at scale? A: Generation! We release the ASIMOV Benchmark for Semantic Safety of robots at asimov-benchmark.github.io @GoogleDeepMind