Oration partners with Soniox: the benchmark numbers behind Indian-language voice AI
We're partnering with Soniox to bring their speech-to-text into Oration. Rather than just describe why, we're publishing the numbers: word error rates by language, how they compare to the two providers most voice AI teams evaluate first, and what that gap actually looks like on a live call.
The problem, in one statistic
Speech recognition models that are trained mostly on monolingual audio don't just get worse on Indian phone calls — they get measurably worse. A 2025 benchmark study on code-switched Hindi-English speech (the HiACC corpus, built specifically to test this) found that ASR models see a relative increase in word error rate of 30–50% when the input is code-switched compared to monolingual speech. That's not a Soniox number — it's an independent academic finding, and it's the reason most STT vendors quietly do worse on Indian calls than their headline accuracy numbers suggest.
On Indian phone lines, code-switching isn't the exception — it's the default. People mix Hindi and English mid-sentence. They read order IDs as letters and digits. They say "pachaas hazaar" instead of "50,000." They do this quickly, on a variable-quality mobile connection, often while already frustrated with the reason they're calling.
When STT gets any of this wrong, the LLM downstream doesn't get a chance to recover. It asks the caller to repeat themselves, pulls up the wrong account, or routes the call incorrectly — and the agent looks broken even when the prompt and the model are fine.
| Problem | What goes wrong |
|---|---|
| Language-switching | Hinglish, Telugu-English, Bengali–English — one language's grammar, another's nouns, in the same utterance |
| Script and transliteration | Output lands in Devanagari, Latin, or a mix depending on how the model interpreted the audio |
| Alphanumerics | Order IDs, phone numbers, PINs, and spoken amounts get truncated or merged |
| Proper nouns | Locality names, brand names, and app names spoken in English come out garbled |
| Telephony conditions | Fast, informal speech on compressed mobile audio — not the clean, read-aloud audio most STT models are benchmarked on |
Most providers handle language-switching with a two-step pipeline: language identification (LID) first, then a language-specific model. That breaks the moment a caller switches languages mid-sentence — LID commits to Hindi, the English chunk gets forced through a Hindi phoneme model, and the transcript your agent receives is already unusable. Soniox uses a single unified model across 60+ languages instead, so there's no LID step to get wrong and no per-language config to maintain.
What the benchmarks actually show
We pulled Soniox's published benchmark comparisons — a 2025 study run across 60 languages on real-world YouTube audio, evaluated against Google Cloud Speech-to-Text — for the six Indian languages our customers run agents in most. Full methodology and per-language reports here.
| Language | Soniox WER | Google WER | Soniox's error rate is roughly |
|---|---|---|---|
| Hindi | 1.25% | 2.84% | 2.3× lower |
| Bengali | 6.3% | 48.9% | 7.8× lower |
| Marathi | 6.9% | 43.4% | 6.3× lower |
| Telugu | 8.2% | 37.0% | 4.5× lower |
| Tamil | 9.6% | 42.3% | 4.4× lower |
| Kannada | 10.9% | 56.6% | 5.2× lower |
(Word error rate — lower is better. Soniox vs. Google, by language.)
Two things stand out. First, the gap isn't small for any of these languages — even the closest case (Hindi) is more than double Google's error rate, and for languages like Bengali and Kannada it's not a close comparison at all. Second, the gap gets wider, not narrower, as you move away from Hindi. That tracks with what most voice AI teams already suspect: STT accuracy in India tends to concentrate around Hindi and English, and everything else is treated as an afterthought. Soniox is trained natively across all of these languages rather than extending an English-first model, which is where that consistency comes from.
For teams that benchmark against Speechmatics instead of Google, the same study puts Hindi at 7.4% for Soniox versus 14% for Speechmatics — a smaller gap than the Google comparison, but still roughly half the errors.
Real-time performance: accuracy and latency together
WER on batch audio tells you whether a model can transcribe accurately. It doesn't tell you whether it can do it fast enough for a live phone call, where the agent has to decide when the caller has finished speaking and respond before the pause starts to feel unnatural. For that, the more relevant number comes from Pipecat's open-source STT benchmark — a third-party, reproducible evaluation (not run by Soniox) of streaming transcription across 1,000 real-world conversational samples, scored on semantic WER (errors that would actually change what a downstream LLM does) and time-to-final-segment (TTFS), i.e. how long after the caller stops talking the final transcript arrives.
| Provider | Semantic WER | Perfect transcripts | Latency (median) | Latency (P95) | Price / hr |
|---|---|---|---|---|---|
| Azure | 1.21% | 82.9% | 1,016ms | 1,345ms | $1.00 |
| Soniox | 1.25% | 84.1% | 249ms | 281ms | $0.12 |
| Speechmatics | 1.40% | 83.2% | 495ms | 676ms | $0.56 |
| Cartesia | 1.47% | 84.2% | 299ms | 328ms | $0.43 |
| Deepgram | 1.71% | 76.5% | 247ms | 298ms | $0.55 |
| AssemblyAI | 1.74% | 83.9% | 335ms | 534ms | $0.57 |
| 2.84% | 69.0% | 878ms | 1,155ms | $0.96 |
Azure edges out Soniox on raw WER in this particular English-heavy dataset, but at roughly 4× the latency and over 8× the price — a real-time voice agent waiting a full second for a transcript feels sluggish regardless of how clean the final text is. Soniox is the only provider in the top tier on both accuracy and speed: sub-300ms at P95, and the highest "perfect transcript" rate (no errors at all) of any provider tested. That combination — near-instant partial transcripts plus low error rates — is what lets turn-taking and intent detection work off the first few words of a sentence instead of waiting for the caller to finish and hoping the correction lands in time.
What this looks like on a call
Lower WER is an abstraction until you see what it prevents. These examples are illustrative — built from the failure patterns that show up across support, collections, and delivery calls, not transcripts of a specific real call.
Collections — Hindi/English, spoken amount
Caller: "Sir mera EMI ka amount thoda zyada dikha raha hai — maine pichle mahine hi teen hazaar paanch sau pay kiya tha, phir bhi bakaya same hai."
Weaker STT: "...maine pichle mahine hi 3050 pay kiya tha..."
Soniox: "...maine pichle mahine hi ₹3,500 pay kiya tha..."
A dropped digit here sends the agent to reconcile against the wrong payment — exactly the kind of error that turns a two-minute call into an escalation.
E-commerce return — Bengali/English, order ID
Caller: "Ami ei product ta return korte chai. Order number ta holo আট নয় দুই পাঁচ ছয়, kintu refund ekhono account e ashe ni."
Weaker STT: "...order number ta holo out non tuple pouch, kintu refund..."
Soniox: "...order number ta holo 89256, kintu refund ekhono account e ashe ni..."
If the order ID doesn't parse, the tool call to look up the return fails silently, and the agent either guesses or asks the caller to repeat five spoken digits a second time.
Delivery tracking — Telugu/English, pincode and landmark
Caller: "Naa order inka randledu. Delivery address Ameerpet, pincode five zero zero zero one six, Metro station daggarlo."
Weaker STT: "...pincode five zero, zero, zero, one, six, metro station dug or low."
Soniox: "...pincode 500016, Metro station daggarlo."
A garbled pincode or landmark means the agent can't confirm delivery status or give a usable ETA — it can only apologize and transfer.
Appointment rescheduling — Kannada/English, proper noun
Caller: "Dr. Prakash jothe appointment reschedule madabeku, next Tuesday matra convenient nanage."
Weaker STT: "Dr. Prakash's oath appointment reschedule mad beku..."
Soniox: "Dr. Prakash jothe appointment reschedule madabeku..."
The doctor's name is the lookup key. Get it wrong and the agent either books with the wrong provider or has to stop and re-ask.
Using Soniox in Oration
On a live call, audio flows from our telephony layer to Soniox; partial transcripts stream into the agent's LLM as the caller speaks, and the LLM triggers tools and sends responses back through TTS.
To turn it on: open any agent, go to Advanced Settings, and select Soniox as the STT provider. Oration supports multiple STT providers — use Soniox when your agents handle Indian languages, heavy code-switching, or calls where getting numbers and names right is critical to the tool calls downstream.
In practice, that means fewer "can you repeat that?" loops, cleaner transcripts, and tool calls that receive the arguments they actually need on the first attempt — which is where WER stops being a leaderboard number and starts being resolution rate.
What's next
The numbers above come from Soniox's published benchmark studies, not from our own production traffic — we're rolling Soniox out across live agents now, and once we have deployment-level data (WER on real call audio, first-call resolution impact, escalation rate) we'll publish that separately with the same level of detail.
If you're building for Indian-language customers, you can select Soniox in your agent settings today, or reach out.
Sources: Soniox speech-to-text benchmarks (Pipecat open-source real-time benchmark) · Soniox vs. Google, by language · Soniox vs. Speechmatics, Hindi · HiACC Hinglish code-switching corpus, PMC (2025)
