AI Providers
Every agent runs on three engines that you choose and plug in yourself: an LLM (the brain that decides what to say), Speech-to-Text / ASR (the ears that hear the caller), and Text-to-Speech / TTS (the voice it speaks with). InfraVoice hosts no models of its own — you bring your own provider keys, and you can mix and match any brain, ears, and voice per agent.
You set all of this up in the dashboard at your-infravoice-host — there's nothing to install and no code to write.
All three (LLM + Speech-to-Text + Text-to-Speech) must be set before an agent can take a live call. Because you supply the keys, you're billed directly by each provider for what your agent uses.
The three engines
| Engine | Tab label | What it does | Popular choices |
|---|---|---|---|
| LLM | LLM Provider | The brain — reads the conversation and decides what to say next | OpenAI, Google Gemini, Anthropic, Groq |
| Speech-to-Text (ASR) | Speech-to-Text Provider | The ears — turns the caller's speech into text in real time | Deepgram, AssemblyAI, Google Cloud Speech |
| Text-to-Speech (TTS) | TTS Provider | The voice — turns the agent's reply into natural-sounding audio | ElevenLabs, Cartesia, Deepgram Aura |
There's also an optional fourth slot, the Analysis LLM, used only for post-call work (summaries, QA scoring, checklist tracking). Leave it empty and the agent's main LLM is used instead — pick a separate, often cheaper, model here if you'd rather run analysis on something lighter than your live voice model.
How to add providers
Everything happens on one screen: open an agent from the Agents page and switch to its Providers tab.
- In the panel for the engine you want (LLM, Speech-to-Text, or Text-to-Speech), choose a Provider from the dropdown.
- Paste the provider's API Key. Depending on the provider, you may also fill in a Base URL, a Model, and — for Text-to-Speech only — a Voice ID. The form shows only the fields that provider needs and pre-fills sensible defaults (for example a recommended model name), so most of the time you only paste a key.
- Click Save LLM, Save ASR, or Save TTS. Your key is validated with the provider before it's stored, so you'll know right away if it's wrong.
That's it — repeat for all three engines and your agent has a brain, ears, and a voice.
Tip: Some voices offer extra tuning. Text-to-Speech panels for providers with preset voices let you pick from a list (or choose Custom… to paste your own voice ID), and a few providers expose an Advanced Options panel for things like speaking rate and quality.
Remember to publish. Saving a provider updates a draft. Use the version banner at the top of the editor to Publish the new version so live calls start using it.
Reuse a key across agents — Credentials
Rather than pasting the same key into every agent, save it once and reuse it everywhere. Under Build → Credentials you'll find Provider Credentials:
- Click Add credential.
- Choose the Type — LLM, Speech-to-Text, or Text-to-Speech.
- Pick the Provider, give it a Name, and paste the API Key (plus optional Base URL, Model, or Voice ID).
- Save. The key is validated before it's stored, encrypted at rest, scoped to your workspace, and never shown in full again.
Back on an agent's Providers tab, a saved credential appears in a picker labelled (reuse across agents) — select it instead of typing a key. Prefer to enter a key inline? You can tick Save to workspace credentials for reuse while saving so it's available to your other agents next time.
Supported provider families
You're not locked into any one vendor — swap providers per agent at any time. The dashboard's dropdowns list every option; here's the landscape at a glance.
LLM (the brain)
| Provider | Notes |
|---|---|
| OpenAI | GPT-4o family — popular, excellent streaming |
| Anthropic | Claude 3.5 Sonnet, Claude 3 Opus/Haiku — strong reasoning |
| Google Gemini | Gemini 2.5 Flash (recommended), 2.0 Flash, 1.5 Pro |
| Azure OpenAI | GPT-4o / GPT-4 via your Azure resource |
| Groq | Llama, Mixtral — ultra-fast inference |
| Cerebras | GPT-OSS — among the fastest inference available |
| DeepSeek | DeepSeek V3, DeepSeek Coder |
| Mistral AI | Mistral Large / Medium / Small — good multilingual |
| Fireworks AI, Together AI | Fast open-source model hosting |
| xAI (Grok) | Grok models |
| OpenRouter | One key, 100+ models via a gateway |
| Perplexity | Search-augmented responses |
| SambaNova, NVIDIA NIM | Enterprise-grade fast inference |
| UltraSafe AI | Proprietary voice-optimised LLM |
| Anyscale, Databricks (DBRX), Lepton AI, OctoAI | OpenAI-compatible endpoints |
| Ollama, LM Studio, vLLM | Self-hosted / local model servers. A Base URL on a private address is only accepted when the deployment lists that host in USF_TOOL_ALLOWED_INTERNAL_HOSTS — on the hosted platform, Base URLs must be publicly reachable. |
| Custom Provider | Any OpenAI-compatible endpoint — you supply the Base URL |
Speech-to-Text / ASR (the ears)
| Provider | Notes |
|---|---|
| Deepgram | Nova-3 — fastest real-time transcription (recommended) |
| AssemblyAI | Universal model — high accuracy |
| Google Cloud Speech | 125+ languages (paste your Service Account JSON in the API Key field) |
| Azure Speech | Real-time — set your region in the Base URL |
| Gladia | Whisper-based, 99 languages |
| ElevenLabs, Cartesia | Real-time streaming transcription |
| USF Mini ASR | UltraSafe AI proprietary real-time ASR |
| Custom Provider | Any compatible speech-to-text endpoint |
Text-to-Speech / TTS (the voice)
| Provider | Notes |
|---|---|
| ElevenLabs | Turbo v2.5 — among the most natural voices |
| Cartesia | Sonic — fastest streaming voice |
| Deepgram Aura | Aura-2 — ultra-low latency |
| OpenAI TTS | TTS-1 / TTS-1-HD — six built-in voices |
| Azure Speech | Neural voices — set your region in the Base URL |
| Google Cloud TTS | Chirp 3 HD voices (paste your Service Account JSON in the API Key field) |
| LMNT | Ultra-fast, low-latency streaming |
| USF Mini TTS | UltraSafe AI streaming synthesis with preset and cloned voices |
| Custom Provider | Any compatible text-to-speech endpoint |
Choosing a good combination
Any brain, ears, and voice will work together — but a few pairings are worth knowing:
| Goal | LLM | Speech-to-Text | Text-to-Speech |
|---|---|---|---|
| Lowest latency | Cerebras | Deepgram | Cartesia |
| Best quality | Gemini | Deepgram / USF ASR | Cartesia |
| Widest language coverage | Google Gemini | Google Cloud Speech | Google Cloud TTS |
| Fully self-hosted brain | Ollama or vLLM | Deepgram | Deepgram Aura |
When you've saved all three, head to Voice Test to hear your agent live in the browser before you put it on a phone number.