Most voice AI teams delay this decision far longer than they should - and eventually hit the same wall. The LLM ends up being the single biggest driver of both latency and cost.

The solutions teams reach for are usually workarounds: • Racing multiple LLM endpoints and taking the fastest response (works, but expensive) • Cron jobs that monitor response times and hot-swap providers • Hosted OSS via Fireworks or Baseten (useful as a bridge, rarely the destination)

However, fine-tuning and self-hosting are no longer hard for modern engineering teams. With frameworks like @UnslothAI and Verl, teams can fine-tune strong models for a few hundred dollars - and regain control over: • Latency • Cost • Data residency / regionality • And (most importantly) Model outputs

The data prep step is also easier than most teams expect. You can reach baseline performance for common call workflows without a perfect dataset, then iterate from there.Evals are non-negotiable. When teams make this shift, we consistently see: • ~330ms P99 time-to-first-token • 50%+ cost reductions vs managed LLM APIs

This typically only starts to make sense at ~15k+ calls per day. But that’s also when customers start noticing. Lower latency shows up immediately in: • Conversation flow • Turn-taking • Perceived intelligence