Speech-to-text (STT), also called automatic speech recognition (ASR), is technology that converts spoken audio into written text. It powers meeting transcripts, live captions, voice typing, voice assistants and call centre analytics. Modern systems use deep learning and reach high accuracy on clear audio, though accents, noise and jargon still cause errors.
This page covers transcription specifically. For recognition in general, including images and faces, see What is AI Recognition?
Key takeaways
- STT turns audio into text; text-to-speech does the reverse.
- Accuracy is usually measured with word error rate (WER).
- Features such as speaker diarisation, punctuation and timestamps make transcripts usable.
- Audio quality has a big effect on accuracy.
- Transcripts feed other AI tasks such as summaries, search and analytics.
Speech-to-text explained in simple terms
Think of a fast, tireless stenographer. They listen to speech, work out which sounds they heard, use their knowledge of language to decide which words make sense, and type them out with punctuation. A good stenographer uses context: they know “recognise speech” is more likely than “wreck a nice beach” in a tech meeting. STT systems do the same with learned models.
How speech-to-text works
- Audio capture and pre-processing: the signal is cleaned and split into short overlapping frames.
- Feature extraction: each frame becomes numbers describing its sound, often a log-mel spectrogram.
- Acoustic and language modelling: a neural network, typically a transformer-based encoder with a decoder or a similar design, maps audio features to text, using language patterns to choose likely words.
- Post-processing: adds punctuation, capitalisation, number formatting and timestamps.
- Optional extras: speaker diarisation (who spoke when), translation and keyword extraction.
OpenAI’s open-source Whisper model (2022), trained on a very large and diverse set of multilingual audio, made robust transcription widely available to developers.
Measuring accuracy: word error rate
WER counts the words a system gets wrong compared with a human reference transcript:
WER = (substitutions + deletions + insertions) ÷ number of words in the reference
Worked example: the reference is “please send the invoice by Friday” (6 words). The system outputs “please send an invoice Friday”.
- Substitution: “the” → “an” (1)
- Deletion: “by” missing (1)
- Insertions: none (0)
WER = 2 ÷ 6 ≈ 33%. Lower is better. Always measure WER on audio similar to your real use, because results on clean benchmark audio can be much better than on noisy calls.
Key features to look for
| Feature | Why it matters |
|---|---|
| Speaker diarisation | Labels who said what in meetings and calls |
| Timestamps | Links text to audio for review and captions |
| Custom vocabulary | Improves accuracy on product names and jargon |
| Real-time streaming | Enables live captions and voice assistants |
| Multilingual and code-switching support | Handles mixed-language speech, common in India |
| Punctuation and formatting | Makes transcripts readable |
Where STT is used
- Meeting notes and summaries.
- Live captions for accessibility and video.
- Call centre analytics and quality monitoring.
- Voice typing and dictation, including medical and legal notes.
- Voice assistants and voice search.
- Searchable archives of podcasts and videos.
What affects accuracy
Background noise, overlapping speakers, microphone quality, strong accents, fast speech, rare terms and mixing languages mid-sentence all increase errors. Improving audio (a good microphone, less echo) often helps more than switching providers.
Limitations
- Errors on names, numbers and domain jargon.
- Weaker performance for some accents, dialects and low-resource languages.
- Some models can invent words during silence or noise, a form of hallucination.
- Recordings may contain personal data, so consent and data handling rules apply.