Performance
Measured, not quoted.
Every number on this page comes from our test suite, the Arena benchmark, production Edge or a live call. Where we have not proven something yet, we say so.
The numbers
| Quantity | Value | How it was measured | Also seen |
|---|---|---|---|
| Engine compute, p99 per 10 ms frame | 46 µs | Offline, 16 kHz, all modules, with echo reference | 50 to 65 µs on production Edge |
| Audio delay added, default stt profile | 5 ms | At 16 kHz (7 ms at 24 and 48 kHz, 8 ms at 8 kHz) | Confirmed in production call summaries |
| Speech-to-text WER, stt profile | 0.233 | Deepgram nova-3 over 110 noisy and echo-heavy test calls | 0.250 with no processing |
| Echo suppression | 33 dB | Live Pipecat call over Daily | 31 to 39 dB on synthetic and real-Opus scenes |
| False interruptions after a backchannel | 0 | Within 2 s of an acknowledgement, live Pipecat | 48 / 48 correct on held-out test voices |
| End-of-turn decision | 360 ms | turn_end p50 after the caller's last word, live | p95 610 ms |
| Barge-in decision with interim transcripts | 355 ms | p50 from speech onset | max 367 ms in the verification suite |
| False barge-ins from echo | 0 | Pure echo delays from 0 to 1,000 ms | Covered by regression tests |
| CPU per call | ≈ 6 ms / s | 5.2 to 6.0 ms of CPU per second of audio, DSP only, measured on Railway | 11 to 16 ms / s with Silero VAD on |
| Edge capacity, Silero VAD on | 7,200 | Concurrent calls: 8 replicas × 900, measured on Railway | ≈ 3.5 M calls a day at 3-minute calls |
| Summary ingest | ≈ 11,000 / s | Control-plane load test with 1 M summaries | About 95× the average of 10 M calls a day |
Speech-to-text
Cleaner audio isn't more recognisable.
On noisy calls, every enhancer we tested raised Deepgram's word error rate against unprocessed audio, our own trained model included. Removing the agent's echo is what helps: 0.274 vs 0.331 on echo-heavy calls. So the stt profile keeps echo removal and only a gentle noise floor, for 0.233 vs 0.250 across all 110 calls.
- No processing 0.207
- Tenms stt 0.211
- Full noise suppression (listening) 0.236
- Tenms trained model 0.240
- GTCRN* 0.279
- DeepFilterNet3* 0.356
- RNNoise* 0.365
- LaCo-SENet* 0.449
WER, lower is better · line = no processing
- No processing 0.331
- Full noise suppression (listening) 0.290
- Tenms trained model 0.282
- Tenms stt 0.274
WER, lower is better · line = no processing
Deepgram nova-3 corpus WER on the Arena's frozen telephony sets (8 and 16 kHz). * Also over the 1 ms / 10 ms budget. On noisy calls, stt and no processing are statistically tied. Each set's 95 % interval is about ±0.05. Sources: MODELS_AND_BENCHMARKS.md, TRAINING_REPORT.md §7.1, LEADERBOARD.md.
Our own model
Trained on a laptop. Shipped one head.
tenms-gru-v1 is one small causal network with four heads: Focus, Turn, Pulse and speaker. Each head ships only if it wins its benchmark. One did.
- ~10 h
- training on one Apple M4 Max, about 4,200 h of simulated calls
- 0.22 MB
- int8 weights; quantisation cost 0.013 dB
- 48 µs
- p99 per 10 ms frame, 5 % of the 1 ms budget
| Head | Ships | Why |
|---|---|---|
| Focus, listening | Ships as listening_mini | 12.1 vs 5.9 dB background removed in pauses, UTMOS 1.545 vs 1.497, at the DSP latency. |
| Focus, speech-to-text | No | Cleaner audio, more errors: WER 0.240 vs 0.207 unprocessed. |
| Turn | Not yet | Best voice detection we have measured (F1 0.803), but end of turn takes 1.16 s against a 600 ms budget, and it caught 78 of 96 barge-ins. |
| Pulse | Not yet | Runner-up: 0.447 vs 0.479 correlation with recognition loss, at a twelfth of the winner's cost. |
| Speaker | Not yet | The head did not learn (EER 0.468 vs 0.426 for DSP). |
Arena results, 2026-11-06. Source: TRAINING_REPORT.md.
How models are chosen
Nothing ships until it wins.
Open models, our own DSP and our trained models compete in the same harness. A candidate replaces a stage of the engine only if it beats what is there, inside the latency budget, on the same audio.
Same audio
Evaluation sets are generated once from fixed seeds and stored with a sha256 manifest that is checked on every load. Half are 8 kHz telephone audio with G.711 or AMR-NB codecs and packet loss; half are 16 kHz wideband.
Same CPU
One machine, every thread pool pinned to one thread, warm-up excluded. Wall-clock time is measured around each model call.
Streaming, causal
Every candidate is fed audio chunk by chunk, behind the engine's 10 ms framing, the way it would run on a call. Latency is measured by cross-correlation, not taken from the paper.
Real ASR
Word error rate comes from a production speech-to-text model, with a cache and a billing ledger so every result can be reproduced.
Decision rule
- Only candidates that actually ran are ranked. Ones that failed or could not be installed are listed with the reason.
- A candidate whose license does not allow shipping is ranked for reference and can never win.
- A candidate is in budget only if every latency and CPU limit was measured and met.
- The winner is the best in-budget, shippable candidate. Ties go to the one that uses less CPU.
Budget per stream, one core
- Per 10 ms frame
- p99 < 1 ms
- Audio-path delay
- ≤ 10 ms
- End-of-turn decision
- mean ≤ 600 ms
Published limits
What we haven't solved yet.
You would find these in a pilot. We would rather you read them here first.
- Music and babble on the agent path
- The default stt profile keeps a gentle noise floor, so background music and conversation mostly pass (babble about +0.5 dB). The listening tiers remove more, for audio people hear.
- Barge-in without transcripts
- Audio alone takes about 1.0 to 1.3 s to confirm a barge-in. With interim transcripts it is about 355 ms.
- End of turn is prosodic
- It reads timing and intonation, not words. A caller reading out digits with pauses can end a turn early.
- Double talk over codecs
- Echo is suppressed rather than cancelled, so caller speech at or below the echo level is attenuated while both talk.
- Pulse scores are heuristics
- MOS and ASR-risk are not yet calibrated against listening tests or measured WER.
- One trained head ships
- The stt and listening profiles are hand-tuned DSP. Our first model ships only its Focus head, as the opt-in listening_mini tier.
Benchmark it on your calls.
Send us recordings from your own traffic. We return the events, the cleaned audio and the per-call summaries, so you can compare against what your pipeline does today.