Voice agents: APIs are not enough—local TTS depends on language and licensing

September 22, 2026 update: GPT‑Live‑1 and Gemini 3.8 Live

A voice agent now involves more than transcription, a text response and speech synthesis. GPT‑Live‑1 was announced on September 10, before this review window; it is included as missing context. Its full-duplex interface separates live conversation from backend reasoning and tools. The model documentation quotes $0.05 per session minute, billed per second, plus backend model and tool costs. One hour therefore costs $3 before the backend.

Google announced Gemini 3.8 Live on September 15 and updated its post on September 17. It describes a standard version, Extended Thinking and background tool work. Developer access, enterprise preview and availability in a consumer app are distinct.

For support, test interruptions, waiting for CRM data, correcting a misheard value and handing over to a person. The model may keep talking while a lookup runs; it must not claim a refund succeeded before the tool confirms it. Measure time to the first response and to the actual result, full-session cost and handover success. The TTS and local options below address other layers of that process.

Custom voices are no longer limited to a single API. Alongside xAI and OpenAI, teams can test local or open-weight TTS models that support zero-shot voice cloning from a short reference recording.

A practical test by Jirka Herník compared Higgs Audio, Chatterbox, and Qwen TTS on his own voice. It is not a laboratory benchmark, but it captures a common production problem: a good demo in a supported language does not guarantee a reliable Czech voice workflow.

What the Czech test showed

Higgs Audio sounded closest to the reference voice in this test. Chatterbox struggled with the speaker's surname, while Qwen broke down on a sentence that switched from Czech to English. Longer reference audio and a clip containing the correct pronunciation did not automatically solve every issue.

This is one voice, a few reference clips, and one configuration—not proof that one model generally beats the others. It is still a useful smoke test.

For a Czech product, test names, abbreviations, amounts, and addresses; Czech-only and Czech-English code-switching; consistency across repeated generations; longer passages; generation speed and VRAM; and unwanted repetitions, stuttering, or speech that was not in the prompt.

Supported language is not a minor detail

The current Chatterbox Multilingual repository lists supported languages, but Czech and Slovak are not included. The official Qwen3-TTS report describes ten training languages, also without Czech. Higgs Audio V2 supports a limited language set.

A model may produce something outside its documented languages. That is not the same as production support. Code-switching can alter accent, pacing, pronunciation, and even speaker identity. If a product must reliably read Czech company names, customer names, or invoice numbers, this eval matters more than a general leaderboard.

Code and weights can have different licences

“Open-source TTS” is too broad a label.

Qwen3-TTS publishes its repository and models under Apache 2.0. Chatterbox uses the MIT licence. Higgs Audio is more complicated: its repository is Apache-2.0, while Higgs Audio V2 weights use the Boson Higgs Audio 2 Community License. It includes attribution requirements and requires an expanded licence for products or services above 100,000 annual active users.

For company deployment, review the source-code licence, model-weight licence, commercial and redistribution terms, attribution, dependency and dataset licences, and rules for fine-tuning or derived models. Licensing can change the model choice and distribution architecture; it is not a checkbox after the prototype.

When local TTS makes sense

Local inference is useful when reference recordings and generated audio need to stay in company infrastructure, when you generate enough volume to avoid per-character pricing, or when a workflow needs predictable latency.

Examples include internal training and product tutorials, podcast or video voice-over drafts, spoken internal reports, a voice layer over a local assistant, batch audio versions of content, and testing before selecting a paid API.

Local is not automatically cheaper. Include GPU time, operations, updates, monitoring, queues, fallbacks, and pronunciation tuning.

APIs and local models solve different problems

xAI supports custom voices in TTS and Voice Agent APIs. OpenAI's Realtime API covers live conversation, tool calls, translation, and transcription. These services manage part of the infrastructure.

Local TTS gives more control over data and runtime, but you must provide orchestration. A real voice agent still needs ASR, interruption handling, dialog state, tools, timeouts, moderation, audit, and escalation to a person.

A hybrid architecture can be sensible: local TTS for internal or batch output, an API for real-time conversation, and one governance layer across both.

How I would evaluate models

Do not test one attractive sentence. Build a small company eval set:

30 ordinary Czech sentences,
20 names, brands, addresses, and abbreviations,
10 sentences with numbers, currencies, and dates,
10 Czech-English switches,
three reference-audio lengths,
three runs per sample.

Human reviewers score intelligibility, speaker similarity, textual accuracy, consistency, latency, and fitness for the actual use case. For customer-facing audio, also test headphones, phone playback, and noisy environments.

Governance starts before cloning

Voice cloning needs explicit consent, an audit trail, retention rules for reference recordings, and a process for withdrawing consent. A synthetic voice must not receive broader authority merely because it sounds trustworthy.

For customer communication, add a clear AI disclosure, limits on what the agent may promise or trigger, and an easy route to a person.

My take

Local voice cloning is now practical. For Czech, however, the decision is not “pick the model with the best demo.”

Supported language, code-switching, weight licences, operating costs, and performance on your own eval set matter. APIs manage infrastructure; local models offer control. Neither path manages consent, audit, and safety for you.

Sources: Jirka Herník: Higgs Audio vs Chatterbox vs Qwen TTS, Boson AI: Higgs Audio, Higgs Audio V2 weight licence, Resemble AI: Chatterbox, Qwen3-TTS, Qwen3-TTS technical report, xAI Custom Voices, and OpenAI voice models in the API.