john so

What is the best way to learn behaviors from videos? By modeling point trajectories, our method, ATM, helps robots learn even from videos of humans and other robots 🤝 Grateful to have worked with amazing collaborators; check it out below!

the six love languages are words of affirmation, quality time, gift giving, acts of service, physical touch, and.. ..machine learning. this year ML valentines have a new home on the internet <3

I spent the past month reimplementing DeepMind’s Genie 3 world model from scratch Ended up making TinyWorlds, a 3M parameter world model capable of generating playable game environments demo below + everything I learned in thread (full repo at the end)👇🏼

@Almondgodd @Tesla_Optimus my goat fr

Over the past 3 months I’ve been interning @tesla_optimus to build AGI for the real world. Robotics is the hardest frontier of AI, but it gives us a clear path to eliminating scarcity. After working on Optimus, I’ve never been more confident that universal abundance is within reach. Incredibly excited to see the sci-fi future our children will be born into.

Your bimanual manipulators might need a Robot Neck 🤖🦒 Introducing Vision in Action: Learning Active Perception from Human Demonstrations ViA learns task-specific, active perceptual strategies—such as searching, tracking, and focusing—directly from human demos, enabling robust visuomotor policies under visual occlusions. 🧵👇

coming to the Exploratorium today! thank you @CatieCuan for bringing me along on this wild adventure :) exploratorium.edu/visit/calendar…

it's a bird! it's a plane! it's a ... 9 foot robot hand?

@CadeGordonML @Anthropic woo congrats big dog!!! 🐕

Excited to share that I'll be joining @Anthropic to work on pretraining science! I've chosen to defer my Stanford PhD, where I'm honored to be supported by the Hertz Fellowship. There's something special about the science, this place, and these people. Looking forward to joining some of my most brilliant and compassionate colleagues!

Introducing ✨Latent Diffusion Planning✨ (LDP)! We explore how to use expert, suboptimal, & action-free data. To do so, we learn a diffusion-based *planner* that forecasts latent states, and an *inverse-dynamics model* that extracts actions. w/ @_oleh @DorsaSadigh @chelseabfinn

Today, we're releasing a new paper – One-Minute Video Generation with Test-Time Training. We add TTT layers to a pre-trained Transformer and fine-tune it to generate one-minute Tom and Jerry cartoons with strong temporal consistency. Every video below is produced directly by the model in a single shot, without editing, stitching, or post-processing. Every story is newly created. Demos: test-time-training.github.io/video-dit/ Paper: test-time-training.github.io/video-dit/asse…

A simple idea to build the @UCBerkeley startup alumni network has grown beyond my wildest dreams into #AccelScholars, a tight-knit community of the most ambitious, talented, kind-hearted people, whose individual stories we’ve been fortunate to support for the past eight years

@1x_tech so quiet and so smooth 😮

Introducing NEO Gamma. Another step closer to home.

@karansdalal @1x_tech wow indeed

@1x_tech Wow

Time to democratize humanoid robots! Introducing ToddlerBot, a low-cost ($6K), open-source humanoid for robotics and AI research. Watch two ToddlerBots seamlessly chain their loco-manipulation skills to collaborate in tidying up after a toy session. toddlerbot.github.io

Can robots leverage their entire body to sense and interact with their environment, rather than just relying on a centralized camera and end-effector? Introducing RoboPanoptes, a robot system that achieves whole-body dexterity through whole-body vision. robopanoptes.github.io

Update on @suchirbalaji We hired private investigator and did second autopsy to throw light on cause of death. Private autopsy doesn’t confirm cause of death stated by police. Suchir’s apartment was ransacked , sign of struggle in the bathroom and looks like some one hit him in bathroom based on blood spots. It’s a cold blooded mu*d*r declared by authorities as suicide. Lobbying in SF city doesn’t stop us from getting justices. We demand FBi investigation @elonmusk @VivekGRamaswamy @GavinNewsom @MEA

Can robots acquire real-world manipulation skills without real-world robot training data? Introducing Im2Flow2Act: — a framework that leverages object flow to bridge human video and simulation data, enabling scalable skill acquisition. Project Website: im-flow-act.github.io

Can my robot cook my food, rearrange my dresser, tidy my messy table and do so much more without ANY demos or real-world training data? Introducing ManipGen: A generalist agent for manipulation that can solve long-horizon robotics tasks entirely zero shot, from text input! 1/N

Can robots learn to manipulate with both care and precision? Introducing Adaptive Compliance Policy, a framework to dynamically adjust robot compliance both spatially and temporally for given manipulation tasks from human demonstrations. Full detail at adaptive-compliance.github.io

We recently launched umi-data.github.io as a community-driven effort to pool UMI-related data together. 🦾 If you are using a UMI-like system, please consider adding your data here. 🤩🤝 No dataset is too small; small data WILL add up!📈