David Chen

@Rajath_DB @livekit @southpkcommons The whole frame. WebRTC on a decent internet can def do <200ms glass to glass so there’s another 100ms budget for inference.

@chenosaurus @livekit @southpkcommons 300ms is brutal once you factor webRTC jitter buffer + a VLM pass. are you splitting it like perception on-device and only sending semantic deltas up, or actually piping frames through inference each tick?

Best way to kick off a week with the @LiveKit robotics team: win the @southpkcommons Embodied AI Hackathon the weekend before. Over 48 hours, we really cooked something special. Distributed low latency inference (VLMs, ACT, MolmoAct 2), teleop, voice agent orchestration, all powered by @LiveKit.

@Rajath_DB @livekit @southpkcommons In some ways it’s more tolerant than voice. If voice takes > 1s to respond, it starts to feel off. If the robot waits an extra second to move, nobody notices.

@chenosaurus @livekit @southpkcommons distributed low latency inference for embodied agents is wild. what was the e2e latency budget you were hitting? voice side i'm usually fighting to keep tts+llm under 800ms, robotics has to be way tighter

@Rajath_DB @livekit @southpkcommons Ideally under 300ms rt for full roundtrip of camera to inference and actions back over webRTC.

@pham_blnh, @chenosaurus and Jacob’s Desktop Robot Assistant looks simple, but the architecture is wild. To move a candy bar, it orchestrates: Voice Agents, VLMs for spatial awareness, ACT policies, and Momo Act 2. The models are distributed across laptops in the room and an H200 server in Finland, executing physical tasks over the internet with real-time, ultra-low latency.

check out my claude knob @trq212 github.com/pham-tuan-binh…

what can i say, my models always get the best data there is so they always work first try built this to test remote hg-dagger style data collection, can’t wait to share the tutorial

@kamathsblog github.com/livekit/client…

@kamathsblog You can use LiveKit to stream from robot to VR.

Getting the Unity stuff working was quite annoying, but worth it. Next up: video streaming back to the headset. Thinking of using WebRTC, or is there a better way? #lekiwi #ROS #Unity

@zoox just expanded their coverage to most of the eastern side of SF!

@jaiselsingh What are some good models that are available today?

@chenosaurus it’s not dependent on any specific camera and we’ve been using a variety of them :)

here’s a video of a perception stack using stereo event cameras to see/track the bullets :)

collect data remotely using @livekit, infer remotely using @livekit excited for what’s to come, handling transport layer for robots should be as easy as setting up a call

You can now play Doom as a world model inside your browser. I’m happy to finally release learning world model learning part 3. pham-tuan-binh.github.io/learning-world… This time we finally go through the entire pipeline and train a “playable” world model. Unlike previous articles, in this one, we’ll go through some more practical notes like: > How to train a model cheaply using spot instances > Identifying when low loss is bad > Loading data from large datasets Hope you like this one. Thanks to my readers for pressuring me into releasing this in time.

@DeoArlo Hell yea. That thing rips!!

Meet my new friend

@tnkrdotai yes

If everything was sorted for you, the parts, the instructions, the code, the community. Would you build a robot this year? Comment YES if you're in 👇 tnkr.ai

@pham_blnh @livekit @LeRobotHF @rerundotio This is the best example of dogfooding your own product. 🤖🦾🐝

used @livekit @LeRobotHF @rerundotio to destroy a wasp nest on my balcony today mission successful, now i can drink my coffee in peace full video in comment

i bet you haven't seen a SO101 mounted on a wall like this before if you want to do the same, here is LeSlider: github.com/pham-tuan-binh… i built it cause i wanted something that can cover my whole desk for tasks like organizing and cleaning i originally wanted to have a belt system like what 3D printers have, but i was too lazy and used a pinion/track with another sts3215 so: > the extra motor shares the same bus as the rest of SO101 > you can have arbitrary length of track > really cheap and easy to assemble and control it turned out better than expected with this, i'm gonna train a model to pick random stuff up across my table and put it into a bin at the end of table (realistically using yolo to scan table, two policies, one for picking up objects, one for dropping)