Benchmarks · Speech-to-Text API
Speed and accuracy, measured from your side.
Every number here is measured the way a customer sees it: audio streamed in real time over the public internet, timed at the client, using the open-source benchmark method behind the public speech-to-text leaderboard. We're preparing our submission for independent testing and will update this page with the official result.
Results: being measured, first publication pending
Where we stand today
Official ranking pending
Headline results
Results by dataset
Lower is better in every column. Speed columns are milliseconds measured at the client; WER is percent.
Results table placeholder until the first run is published
| System | WER % | First word P50 ms | First word P95 ms | Final P50 ms | End of turn P50 ms | False words / hour |
|---|---|---|---|---|---|---|
| Saidwire v1 | — | — | — | — | — | — |
| [Provider, model, settings] | — | — | — | — | — | — |
| [Provider, model, settings] | — | — | — | — | — | — |
Method
- Stream like a live caller. Each clip is sent over the provider's public streaming WebSocket in real time, 20 ms frames, one stream per clip, from a cloud server on the US east coast. Every provider gets the same audio bytes at the same pace.
- Time at the client. We log when each audio frame leaves the client and when each message arrives. Network time to and from the provider is included, as it would be for you.
- Align words to the audio. Reference transcripts are forced-aligned to the audio, so we know when each word ends in the clip.
- Default settings, stated. Each provider runs its documented streaming model for English with interim results on and endpointing at 300 ms (or the nearest supported value). The exact settings appear next to each row.
- Same scoring for everyone. One text normaliser and one scorer for all systems. The normaliser and scorer are published with the results.
- Repeat. Each clip runs [N] times on different days; we report the median run.
Metrics
- First word latency (P50, P95)
- For each reference word: the time from when the client finished sending the audio up to the end of that word, to when the word first appears in any message from the provider (interim, early word or final). Words never output are counted as errors in WER and excluded from latency.
- Final latency (P50)
- The same, to the first final transcript that contains the word. This is the number to compare when your app only acts on finals.
- End-of-turn latency (P50)
- From the end of the last word of a turn to the provider's end-of-turn signal, at the endpointing setting shown.
- Word error rate (WER)
- (substitutions + deletions + insertions) ÷ reference words, on final transcripts after normalisation (lower case, punctuation removed, numbers written one way, filler words such as "um" removed from both sides).
- False words per hour
- Words output on audio with no speech in it (music, crowd noise, hold tones), per hour of that audio.
Datasets
| Dataset | What it tests | Audio | Clips | Hours |
|---|---|---|---|---|
| LibriSpeech test-clean, test-other | Read speech; the common reference point | 16 kHz | — | — |
| EdAcc test | Conversational English across many accents | 16 kHz | — | — |
| Earnings-21 / Earnings-22 | Earnings calls: long-form, finance vocabulary, accents | 16 kHz | — | — |
| Phone condition | The clips above at 8 kHz μ-law, as a phone line delivers them | 8 kHz | — | — |
| Live speech set | Public press conferences and speeches, hand-checked transcripts. We publish the event list and timestamps, not the audio. | 16 kHz | — | — |
| No-speech set | Music, crowd noise and hold audio, for false words | 16 kHz / 8 kHz | — | — |
Our live-event product
Separately from the API, Saidwire covers live events itself. On those live events (October 25 to 27, 2026), words were written about 0.1 s after the audio reached us, usually before the speaker had finished the word; about one word in ten took up to 0.8 s. That figure is for words tracked during live events, measured on our servers, and is not an API result. API results are the tables above.
Caveats
- Our test, not an independent audit. We run it and we have an interest in the result. That's why the clips, settings and scripts are public: please rerun it, and tell us if you get different numbers.
- Your audio will differ. Accuracy depends heavily on microphones, accents, cross-talk and vocabulary. Benchmark on your own audio before you choose; trial credit is there for that.
- Network distance matters. Latency includes the round trip from our client to each provider. A provider with servers nearer to you may be faster for you than it is here.
- Settings change results. Endpointing, interim results and formatting options trade speed against accuracy. We list the settings we used; other settings give other numbers.
- WER depends on the normaliser. Published WER figures from different sources are not comparable unless the normaliser matches.
- Public datasets may be in training data. Any system, ours included, may have been trained on audio from public datasets, which flatters its scores. The live speech set is held out for that reason.
- Point in time. Providers update models often. Every result is dated; we rerun when a provider announces a new streaming model.
- English only. Our API is English only today, so we only benchmark English.
Run it yourself
Ask for an API key with trial credit, choose "Benchmark or leaderboard", and we'll send the scripts and help you set up. Benchmarking Saidwire and publishing the results is allowed under our API terms.