kwindla

NVIDIA Nemotron 3 Super launches today! We've been building voice agents with Super's pre-release checkpoints and running all our various tests and benchmarks. Nemotron 3 Super matches both GPT-5.4 and GPT-4.1 in tool calling and instruction following performance on our realtime conversation, long context, real-world benchmarks. GPT-4.1 is the most widely used LLM today for production voice agents. So an open model that performs as well as GPT-4.1 on hard, voice-specific benchmarks is a big deal. (Side note: we don't think a benchmark "tells the story" about a model's voice agent performance unless it tests model correctness across at least 20 human/agent conversation turns.) The Nemotron models are *fully* open: weights, data sets, training code, inference code. Nemotron 3 Super is 120B params, with a hybrid Mamba-Transformer MoE architecture for efficient inference. You can run it on NVIDIA data center hardware or on a DGX Spark mini-desktop machine. 1M token context. Blog post with full benchmarks, thinking budget notes, inference setup on @Modal, and where we think this goes next. 👇

Nemotron 3 Super live stream Tuesday. I will be watching.

Great piece on the gaps in our developer tooling as we (humans) move up to higher levels of abstraction, and the agents we manage make higher-level decisions than our tools ever have before.

So we can copy by reference arbitrary blocks of layers, wire up KV cache, and maybe train adapters. Bigger models without full VRAM penalty. If we’re lucky, perhaps layer block “circuit edges” align with attention layer patterns in GDN or Mamba hybrid architectures and the KV cache increase is small, too? Really nice work, @dnhkng!

Looking forward to talking about infrastructure and tooling for voice and realtime AI on Thursday evening. If you're in SF and interested in voice agents, multi-modal models, low-latency infrastructure, or agent testing, please join us!

@ShriramKMurthi He wrote/talked about it in the essays and interview book Doubling the Point!

TIL that age 22, JM Coetzee, armed with degrees in english and math, moved to England to work as a *programmer*. Thank goodness that didn't last!

@Yampeleg Say more?

MiniMax-M2.7 woww

Come by and see @EvanGrenda at the AWS booth at GTC. @tavus video avatars, voice agents built with NVIDIA Nemotron models, and new realtime AI architecture patterns in @pipecat_ai!

If you’re at GTC and interested in inference speed, go see @charles_irl talk on Tuesday.

@joelgascoigne Not exactly what you’re asking for, but a lot of people use this voice MCP server with Claude Code. Run Claude Code in tmux, start the MCP server when you want to do remote voice input. Connect via WebRTC from mobile browser. github.com/pipecat-ai/pip…

I feel like there needs to be an iOS ssh client that has voice-first input. I'd love to keep my Claude Code sessions going via mobile just by talking. Does this exist yet?

@altryne @ryancarson You covered way more stuff than I summarized. I just pulled out clips about some of the things that are most on my mind, these days. The fruit fly stuff is wild, but also way outside what I understand!

@kwindla @ryancarson Great summary! Thanks for posting Kwindla!

Sunday podcast catching up. ThursdAI this week on coding agents and open weights models: - @ryancarson used a billion tokens in 24 hours. Every single engineer I work with now lives in Claude Code and Codex. This is a complete change in how we do engineering, and it happened unbelievably quickly. - @WolframRvnwlf talked about something I've been thinking about a lot lately: using open weights LLMs today feels a lot like using Linux in the 90s. Nothing ever just works in a straightforward way; everything's fragile and futzy; capabilities are behind the best commercial models. But, equally clearly, these open weights models are going to be a big part of the future of computing. - @llm_wizard gives a great rundown of the new NVIDIA Nemotron 3 Super model (and the open weights, open data, open source training and inference code approach that is the broader Nemotron ecosystem). Claude made this clip sequence for me, with basically just one prompt and a couple of small refinement requests from me, using my `skill-caption-clip` skill that Claude wrote a few weeks ago.

@GokulEpiphany Tavus!

@kwindla Is this tavus? Or something else?

I'll be at NVIDIA GTC next week talking about voice agents, realtime AI for robotics, and conversational video agents. - Best practices for deploying enterprise voice agents at scale on AWS. - How to design for use cases like customer support, outbound phone calls, and multi-channel UIs that combine voice and text conversation. - Why all production voice agents are becoming "multi-agent systems". - What's coming next. Hang out with @EvanGrenda and me at the AWS booth on the show floor. I'm also hosting several invite-only events. Tell me in the comments what you're planning to do at GTC, and I'll message you!

@lina_colucci The future is multi-modal, multi-model, multi-platform ...

@kwindla Multi-channel UIs is a super interesting topic

@askjohngeorge Me neither!

@kwindla Interesting. Didn't know ctrl+s

TIL

@hampsonw Changed that line in the README.md to fix my mistake. And added a note about the model.

Oh, you're right. So sorry. I must have gotten confused when I edited the README.md. It looks like the 27b row there is supposed to be non-thinking. And the 4s cutoff is obviously kind of arbitrary. The right thing to do, clearly, is figure out some low hanging fruit optimizations to get 27b down below that 4s number. :-) Thank you for figuring that out.

Hoisting this up to a top-level thread because I'd like advicea bout Qwen3.5 27B ... I'm still figuring 27b. I *want* to talk more about it, because it's clealy a good model in a bunch of ways. But it falls into a middle category that's not super useful for me. Maybe skill issue on my part. But: 1. So far I don't have a vLLM/SGLang configuration with a TTFT low enough for the conversational loop part of voice AI. With thinking disabled it's not good enough at tool calling. With thinking enabled, TTFT to first non-thinking token is >1,000ms. 2. It does not do well on the sub-agent tasks I'm most interested in, which are long, multi-turn, and include structured data inputs.

@hampsonw I'm curious what turn times you saw when you ran the benchmark?

@kwindla I'm serving it myself off 2x3090's I ran Qwen3.5-27-GPTQ-INT4

@ekryski 100% with you that a good harness is critical to making all models work for non-trivial tasks. And especially so for non-frontier (or small, or whatever you want to call them) models.

@kwindla Totally. This is why I have a ton of tests in my custom harness, use it every day for real stuff to see what breaks, and have a benchmark suite of all the stuff I’m trying so that I can see which models pass my own real world benchmarks. I bench while I sleep.