Pipecat AI

232 posts

Pipecat AI banner
Pipecat AI

Pipecat AI

@pipecat_ai

100% open source framework for realtime voice and multimodal AI. Maintained by @trydaily engineering team with support from the Pipecat developer community.

Katılım Mayıs 2024
3 Takip Edilen5.2K Takipçiler
Sabitlenmiş Tweet
Pipecat AI
Pipecat AI@pipecat_ai·
Thank you to our community, and all the Pipecat developers out there. You guys are amazing 🫡
kwindla@kwindla

Big day today. Pipecat version 1.0. Two years in the making. The most widely used framework for voice agents, but not just voice agents. Pipecat is a framework for realtime, multi-modal, multi-model AI applications. Contributions from NVIDIA, all the foundation labs, AWS, GCP, and Azure. Used by thousands of startups, scale-ups, and enterprises. Pipecat Subagents v0.1.0. A new library for sub-agent orchestration. Which is just a fancy way of saying running lots of inference loops in parallel, with partially shared context. The basic architecture of Pipecat Subagents is an event bus that works locally, and over the network. And Gradient Bang. The side project that broke containment. Built with Pipecat and Pipecat Subagents. Gradient Bang was actually the proving ground for the early Subagents work. But ... it's also a really fun game.

English
0
3
10
2K
Pipecat AI retweetledi
kwindla
kwindla@kwindla·
Local native-audio voice agent running on an RTX 5090. - @NVIDIAAI Nemotron 3 Nano - audio|text ➡️ text - patched vLLM to implement complete turn prefix caching - ~125ms TTFT - @kyutai_labs Pocket TTS - text ➡️ audio - Nemotron Speech ASR - streaming audio ➡️ text - @pipecat_ai Smart Turn end-of-utterance - ~500ms total voice-to-voice latency - runs bash via tool calls If you're interested in voice and realtime multi-modal AI, come join us at the SF Voice AI Meetup on Thursday May 7th. Talk to engineers from NVIDIA, Kyutai, and Pipecat about what you're building! Links to meetup registration, code, and models on @huggingface below ...
English
9
19
133
7.7K
Pipecat AI retweetledi
kwindla
kwindla@kwindla·
Voice AI Meetup, Thursday May 7th. This one's a special crossover event. T-Bot, who hosts the global Voice AI Spaces meetups, is visiting San Francisco and will MC! - NVIDIA researchers will present some of their really cool recent work on speech models. - We'll have demos and two fireside chats, featuring new developments in models and evals, with @GradiumAI, @ArtificialAnlys, @ServiceNow, and @pipecat_ai. - And, of course, 🍕 and great conversation. - Thanks to the @trychroma team for hosting in their wonderful office/event space. Registration link below. Come hang out with 150 old and new friends!
kwindla tweet media
English
2
7
39
6.8K
Pipecat AI retweetledi
smallest.ai
smallest.ai@smallest_AI·
Smallest AI is now natively supported in @pipecat_ai Lightning TTS + Pulse STT can now plug directly into your Pipecat voice agent pipeline. Docs below ⬇️
English
8
20
100
377.3K
Pipecat AI retweetledi
Tarush Agarwal
Tarush Agarwal@tarush_agarwal_·
We just made Pipecat testing a lot easier. With @cekuraAi + @pipecat_ai , you can now get: • full traces • every tool call with inputs + outputs • complete transcripts with timestamps • mock tools so agents don’t hit live APIs • chat + WebRTC testing, all in one place Everything in one place for both test runs and production debugging. Docs below 👇
English
3
5
20
5.8K
Pipecat AI retweetledi
Pipecat AI
Pipecat AI@pipecat_ai·
Thank you to our community, and all the Pipecat developers out there. You guys are amazing 🫡
kwindla@kwindla

Big day today. Pipecat version 1.0. Two years in the making. The most widely used framework for voice agents, but not just voice agents. Pipecat is a framework for realtime, multi-modal, multi-model AI applications. Contributions from NVIDIA, all the foundation labs, AWS, GCP, and Azure. Used by thousands of startups, scale-ups, and enterprises. Pipecat Subagents v0.1.0. A new library for sub-agent orchestration. Which is just a fancy way of saying running lots of inference loops in parallel, with partially shared context. The basic architecture of Pipecat Subagents is an event bus that works locally, and over the network. And Gradient Bang. The side project that broke containment. Built with Pipecat and Pipecat Subagents. Gradient Bang was actually the proving ground for the early Subagents work. But ... it's also a really fun game.

English
0
3
10
2K
Pipecat AI retweetledi
kwindla
kwindla@kwindla·
Sub-agents in (latent) space! We’ve been working on a side project. As far as I know, this is the first massively multiplayer, completely LLM-driven game. Come play Gradient Bang with us. See if you can catch me on the leaderboard. This whole thing started because I wanted to explore a bunch of things I’m currently obsessed with, in an application of non-trivial size, that felt both new and old at the same time. So … a retro-style space trading game built entirely around interacting with and managing multiple LLMs. Factorio, but instead of clicking, you cajole your ship AI into tasking other AIs to do things for you. Some of the things we’ve been thinking about as we hack on Gradient Bang: - Sub-agent orchestration - Partial context sharing between multiple LLM inference loops - Managing very long contexts, and episodic memory across user sessions - World events and large volumes of structured data input as part of human/agent conversations - Dynamic user interfaces, driven/created on the fly by LLMs - And, of course, voice as primary input If you’ve been building coding harnesses, or writing Open Claw agents, or doing pretty much anything that pushes the boundaries of AI-native development these days, you’re probably thinking about these things too! This is all built with @pipecat_ai, the back end is @supabase, the React front end is deployed to @vercel, and all the code is open source.
English
139
265
2.6K
452.3K
Pipecat AI retweetledi
Daily
Daily@trydaily·
Today's @NVIDIA Nemotron 3 Super launch is an exciting development for voice AI developers. We’re proud to be a launch partner, with day-0 @pipecat_ai support. Developers now have a meaningful open stack for realtime voice, with @NVIDIAAI — Nemotron 3 Nano, Nemotron Speech ASR, Nemotron 3 Super. Open models, open training data. Review how Nemotron 3 Super matches proprietary models in our long-conversation voice agent benchmarks. Happy building, with open source!!
Daily tweet media
English
2
2
21
1.7K
Pipecat AI retweetledi
kwindla
kwindla@kwindla·
NVIDIA Nemotron 3 Super launches today! We've been building voice agents with Super's pre-release checkpoints and running all our various tests and benchmarks. Nemotron 3 Super matches both GPT-5.4 and GPT-4.1 in tool calling and instruction following performance on our realtime conversation, long context, real-world benchmarks. GPT-4.1 is the most widely used LLM today for production voice agents. So an open model that performs as well as GPT-4.1 on hard, voice-specific benchmarks is a big deal. (Side note: we don't think a benchmark "tells the story" about a model's voice agent performance unless it tests model correctness across at least 20 human/agent conversation turns.) The Nemotron models are *fully* open: weights, data sets, training code, inference code. Nemotron 3 Super is 120B params, with a hybrid Mamba-Transformer MoE architecture for efficient inference. You can run it on NVIDIA data center hardware or on a DGX Spark mini-desktop machine. 1M token context. Blog post with full benchmarks, thinking budget notes, inference setup on @Modal, and where we think this goes next. 👇
kwindla tweet media
English
14
34
236
20.2K
Pipecat AI
Pipecat AI@pipecat_ai·
New STT realtime release: @AssemblyAI Universal-3-Pro, supported in Pipecat
AssemblyAI@AssemblyAI

Real-time transcription just got a significant upgrade. Universal-3-Pro is now available for streaming — bringing AssemblyAI's most accurate speech model to live audio for the first time. Developers building voice agents, live captioning tools, and real-time analytics pipelines now get three things they've been asking for: 🔹 Best-in-class word error and entity detection across streaming ASR benchmarks 🔹 Real-time speaker labels — know who said what, as it happens 🔹 Superior entity detection for names, places, orgs, and specialized terminology in real-time 🔹 Code-switching and global language coverage built-in

English
0
4
23
2.9K

Keşfet

@NVIDIAAI @kyutai_labs @huggingface @GradiumAI @ArtificialAnlys @ServiceNow @trychroma @DeepgramAI