The flip side of TTS — STT (speech-to-text) — also lives in mlx-audio. The dominant family is Whisper and its derivatives; mlx-audio ships MLX-optimized loaders for Whisper-class models that take an audio file in and return transcribed text out.
The quality story for STT is more mature than TTS. Whisper-large-v3 (and its Apple-Silicon-friendly distillations) is genuinely strong on most languages, and the transcription latency on M-Pro and up is well below realtime — you can transcribe an hour of audio in minutes.
Install + minimum loop
Same install (mlx-audio package). The Python loop loads a Whisper-class model, points it at a WAV/MP3/M4A file, and gets back text plus per-segment timestamps if you want them.
What you get back
STT typically returns more than just text. The structured response usually includes segments (chunks of audio with start / end times), language detection, and per-token confidence scores if the model exposes them. Use the segments when you need to align text back to audio (subtitles, search-by-time); use just the joined text when you only want a transcript.
Realtime factor
The single most useful number for STT performance is the realtime factor — how many seconds of audio you can transcribe per second of wall-clock time. On an M-Ultra with a Whisper-large MLX model, expect a realtime factor in the 5–15× range for typical audio. On an M-Pro, more like 2–5×. Realtime factor < 1 means you can't keep up with a live stream; realtime factor > 1 means you can transcribe live audio plus have headroom.
The diarization gap
STT (Whisper-class) gives you what was said, not who said it. Speaker diarization — labeling each segment with a speaker id — is a separate capability and isn't yet first-class in mlx-audio as of 2026-05. If you need diarization, the typical workflow is to run STT in mlx-audio for the transcripts and pyannote-audio (or similar, on PyTorch) for the diarization, then merge the two streams. Worth knowing about; not a single-tool problem on MLX yet.
Code
Transcribe an audio file with a Whisper-class MLX model·python
# Re-verified 2026-08-07 on mlx-audio 0.4.7.
# The old `from mlx_audio.stt.generate import generate` no longer exists;
# the entry point is `generate_transcription`. Two of the parameters were
# renamed with it (`audio_path=` -> `audio=`, `model_path=` -> `model=`);
# `output_path` kept its name but changed meaning — it is now a stem, and
# the extension comes from `format`.
from mlx_audio.stt.generate import generate_transcription
result = generate_transcription(
model="openai/whisper-tiny", # see the warning below on WHICH repo
audio="path/to/audio.wav",
output_path="transcript", # extension comes from `format`
format="txt", # txt | srt | vtt | json
)
# `result` is an STTOutput with:
# .text — full transcribed text
# .segments — list of {id, start, end, text, tokens, ...} per segment
# .language — detected language code (e.g. 'en', 'ko')
print(result.text[:500])
Measure the realtime factor on your machine·python
import time, wave
from mlx_audio.stt.generate import generate_transcription
audio_path = "path/to/audio.wav"
# Total audio duration via wave module (works for WAV; for MP3/M4A use ffprobe)
with wave.open(audio_path, "rb") as wf:
duration_sec = wf.getnframes() / float(wf.getframerate())
t0 = time.perf_counter()
result = generate_transcription(model="openai/whisper-tiny", audio=audio_path)
elapsed = time.perf_counter() - t0
print(f"audio duration : {duration_sec:.1f} s")
print(f"transcribe wall : {elapsed:.1f} s")
print(f"realtime factor : {duration_sec / elapsed:.1f}x")
# Measure a REAL file, not a toy. Verifying this lesson on 2026-08-07 I fed it
# a 3.8 s clip and got 0.4x, which says nothing about throughput — model load
# dominated. Load once, transcribe something minutes long, and time only the
# transcribe call if you want a number you can plan capacity with.
Pick a representative audio sample from your actual workflow (a podcast you listen to, a meeting recording, anything 5–15 minutes). Run STT on it with the realtime-factor measurement. Compare to your gut — is the result better or worse than you expected? If you have a multilingual or noisy sample, run that too and note how the realtime factor and quality both change. Two sentences on what you'd plan capacity for.
Progress
Progress is local-only — sign in to sync across devices.