dsa

Today is a day I’ll never forget. I grew up in Cupertino. My dad was a tech founder in the 80s/90s. I was in YC S07. LiveKit is my 5th company. The first 4 didn’t work out. I’ve had a lot of advantages — it still took 20 years to get here. Founders: keep taking shots.

Some great lessons in building a performant real-world voice AI application.

Everyone's asking what AI can do for their job. We asked if it could teach a 6-year-old. Introducing Ello 2.0: reading, math & more for ages 4-9. Free tier for all. If we want AI to force-multiply humanity, let's start with teaching our youngest.

LLMs that are great for coding agents aren't great for voice agents. So today we're launching our first hosted model: Gemma 4, optimized for time-to-first-token without sacrificing task completion rates. Fable/Opus/GPT 5.X optimize for intelligence over speed. But a voice agent handling a customer support call doesn't need PhD-level intelligence. It needs to be smart enough, and it needs to be fast. We verified this across thousands of real-world simulations before shipping.

Apple hasn't fixed this problem since 2011. Using Siri is slower and more painful than using your browser. The UX isn't clear. Voice AI does the job that Siri was supposed to do: Tackle your tasks and have human-like interactions. Siri still feels like a transaction. @dsa @livekit

💫 @LemonSliceAI + @livekit inside

@mitchellh They copied all they could follow, but they couldn't copy my mind, And I left them sweating and stealing a year and a half behind. — Rudyard Kipling

Voice AI has a benchmarking problem. Everyone claims their end-of-turn model is the best, but you couldn't actually compare them. Datasets are proprietary, methods are opaque, and there is no shared ground truth. That changes today. We hit this while developing Turn Detector v1, so we open-sourced eot-bench. 5,000+ real user conversation turns across 14 languages, an evaluation harness that measures the real production tradeoff between latency and false cutoffs, and a live public leaderboard. This should become the default way we evaluate turn detection models.

@tonyzzhao First they ignore you, then they laugh at you, then they fight you, then you win. — Gandhi

Reviews of the original ALOHA paper (2023): "It is very hard to find any strength in a paper that forgets about 50 year of development of robotics... the authors are referred to any good book of control theory or robotics to integrate their background... the solution proposed is just damaging the ongoing discussion on low cost-high performance systems."

We shipped LiveKit Turn Detector v1. Instead of reading transcripts, it listens to speech directly, combining semantic and acoustic cues into one end-of-turn prediction. The result: high accuracy, low latency—the best model we tested across 14 languages. Available on LiveKit Cloud.

"If you had a great tutor, you wouldn't sit down and say please write my paper for me." - @stevenbjohnson The first generation raised on ChatGPT booed AI at graduation. Three founders unpack why, plus Apple's brand new Siri, NotebookLM's biggest update, and whether anyone "wins" the agent race. This week @Jason covered these topics and more with: • Jeffrey Quesnelle (@theemozilla) building Hermes Agent at Nous Research @NousResearch • Steven Johnson (@stevenbjohnson) Editorial Director of NotebookLM @NotebookLM and @GoogleLabs • Russ d'Sa (@dsa) building the voice infra behind ChatGPT, Tesla & Grok at LiveKit @livekit 7 moments worth your time: 1:40 Hermes Agent and the open source race 7:32 Why AI got booed at graduation 10:48 LiveKit powering ChatGPT, Tesla, Grok 27:50 The massive Siri update 52:49 "Functional AGI, unevenly distributed" 57:24 Brittle to brilliant: agents that now just work 1:02:00 Tokenmaxxing and the $1M a year engineer

Operating a robot over the internet means camera frames and joint state arrive at different times, so your observations drift and training data gets misaligned. LiveKit Portal fuses them back together with the same code, whether the robot's in the next room or another continent.

Introducing the LiveKit C++ SDK. Realtime audio, video, and data tracks for C++ apps, with the same low-latency transport our other clients use. Built for the C++ stacks behind robotics, autonomous vehicles, and high-performance media pipelines. livekit.com/blog/livekit-c…

@ItsSnibby @livekit 🫡

@dsa @livekit real-time systems for physical AI have always been bottlenecked by transport layers, so moving beyond audio/video semantics into raw sensor streams feels like an important infra step. @dsa follow back? let’s keep the loop

Today @livekit launched Data Tracks. Physical AI and robotics applications need low-latency, realtime transport for data beyond just audio and video. Data tracks let you transmit binary frames from any source: IMUs, LiDAR, RGBD cameras, control systems with no codec overhead and the same low-latency semantics as media. They support full end-to-end encryption and every frame includes a timestamp, so you can easily align data from different sensors. Excited to see what folks build with this! youtube.com/watch?v=Ju9Jz0…

we won the embodied ai hackathon at @southpkcommons last week this is how we did it 🧵

@lilyjclifford we gonna win this one! wooooooo!!! ✊

our latest speech model CODA is now live and in public beta. we believe this is the last text-to-speech model you'll ever need to build with. why? because it truly crushes every competitive model on both perceptual and brass tacks metrics like latency and throughput. and we are super proud that every day people on the street agree!!

Your outbound phone agent has 1-2 seconds to figure out if it's talking to a person, a voicemail, or an IVR. We shipped Answering Machine Detection (AMD) in LiveKit Agents to do that for you so your agent knows when to keep talking, leave a message, use the keypad, or hang up.

💫 @livekit inside

The co-founder of a $1B VoiceAI infra company says you can't test agents the same way you test traditional software. According to @dsa, you have to test these things almost like you test human beings. Like college degrees, resumes, job interviews, reference checks, etc. What you're really trying to do is build statistical confidence that the person you're hiring can do the task with 99% precision/confidence. Have to test agents the same way by running thousands of end-to-end simulations. Permute accent, language, system prompt, instructions, etc, and see how it performs against the success criteria spit out by your observability stack. This way, you're building confidence, deploying it, and observing which bugs/issues require tweaking. Take those back and make changes to the agent code. Test again, run simulations, and make sure there's no regression. Then deploy again, scale, and observe. That's how the loop goes.

Thanks for having me on, @jason!