Justin Uberti

We’ve integrated the ChatGPT Voice “orb” into the chat view in our latest iOS release, and you can toggle between integrated/fullscreen view. I find chatting with the orb feels more engaging, like there’s an actual thing you’re talking to. Thoughts?

@jezell @stevendcoffey The need makes sense, just wondering if you can approximate it by continuing to append results to the same call.

@juberti @stevendcoffey Imagine for example a tool call which runs a docker build. It’s a very long process. You actually want it to return results incrementally, not the whole chunk at the end for both context length issues and general agent awareness issues.

@stevendcoffey @juberti I think there's a really big gap in the Responses API right now in that there is zero support for streaming tool calls. Now that there are websockets and things are moving more and more to be realtime with voice models on the way that change the interruptions model, the whole tool calls need to be request / response interactions thing is gonna get old fast. Sure, you can make some tools to do things like spin up background terminals and grep over their logs, but real systems are full of realtime events and signals and the current APIs just don't map to that world. I'm looking forward to whatever is the future where the Realtime API + Websocket Responses API are unified into something that can handle those workloads.

GPT-5.4

Agreed. I think of this year-by-year: 2022: GPT 3.5 2023: GPT-4 2024: Reasoners 2025: Coding agents

@SantiagoAfonso @athyuttamre We hope so too 😅

@athyuttamre @juberti My assumption here is that you cannot bring 5.2 xhigh (2-6 minutes of thinking) or pro level intelligence (10+ minutes thinking for the easiest answers) to realtime in the next 36 months. I hope I'm wrong!

We’ve integrated the ChatGPT Voice “orb” into the chat view in our latest iOS release, and you can toggle between integrated/fullscreen view. I find chatting with the orb feels more engaging, like there’s an actual thing you’re talking to. Thoughts?

@wayne_culbreth Realtime API models are separate from the ChatGPT Voice models since 3p customers have somewhat different needs

@juberti When will realitime-1.5 be on iOS?

gpt-realtime-1.5 is the best native audio model on the Scale AudioMultiChallenge benchmark -- this is a significant jump in capability by this measure. There are models that outperform it but they are reasoning models without native audio output.

@adamac hmm. It should be instantaneous. That said, we are working on a number of optimizations to improve this flow (including the smoothness of the UX)

@juberti In a new chat or the same chat. Doesn’t help that the animation is janky as that captures my attention.

@adamac Even in a new chat?

@juberti The time it takes to connect (spinner) is a hindrance to using voice mode imo. Seems like it’s at least 3 seconds every time even when toggling back and forth. Why isn’t this instantaneous?

@fabrohl @LearnAI_MJ Yep! Or just tap it.

@LearnAI_MJ @juberti You drag the circle conversation bubble up into the main view.

interesting! a bit uncanny but definitely shows that some amount of singing with gpt-realtime-1.5 is possible...

Yeah. This is the number one AVM complaint I hear these days. The amazing progress from reasoning models has made the text/audio intelligence delta much more obvious.

You can easily make the model whisper via prompting. Singing is another story, as there are nontechnical restrictions there. You can experiment at platform.openai.com/audio/realtime

@BenjieMalinao hello-realtime is using "marin"

@juberti whats the voice name?

Just released gpt-realtime-1.5 with improved intelligence, instruction following, and voice quality. Try it out today at hello-realtime.val.run (or call 425-800-0042)!

APIs remain the same, so this is a seamless upgrade for existing gpt-realtime users.

@davezatz can you give a more specific example?

@juberti 15 mins today, no issue. Not that you asked, the vocalized and non-vocalized pauses are off-putting in some way I can't put my finger on. :)

Slightly more convenient than hitting the action button to speak to Grok while commuting, as I do now. (ChatGPT and Gemini interrupt themselves over car speakers, so Grok is currently the only option.)

Congrats to LiveKit on their Series C! There's enormous potential for Voice AI, as anyone who's had to wait on hold for support can immediately understand.