Four Ways to Hear

"On Apple Silicon you have four good ears. Pick the default for your voice, keep the others as adapters."

The Non-Negotiable: On-Device

Before comparing engines, the one rule that eliminates most of the market: the engine must run on the machine. A cloud API might be more accurate, but it breaks the two promises that justify Firekeeper — audio staying local and working offline. So every engine below is on-device. The choice among them is about speed, language handling, and packaging, never about "should the audio leave?" The answer to that is always no.

The Four Engines

Engine                 Nature            Role in Firekeeper
------                 ------            ------------------
WhisperKit             Swift-native,     DEFAULT. Handles mixed ko/en.
                       Core ML           large-v3-turbo model.
Apple SpeechAnalyzer   macOS 26 API,     Fast 2nd engine. Zero model
                       zero-download     files. Streaming-friendly.
whisper.cpp            C/C++, Core ML    Fallback / benchmark harness.
MLX Whisper            Python, MLX       Optional batch 'power mode'.

WhisperKit is the default because it's Swift-native, Core ML-optimized, and — critically — it handles Dad's mixed Korean/English speech. Apple SpeechAnalyzer (macOS 26) is the fast second engine: no model files to manage, native streaming, but a language trap we'll hit in two lessons. whisper.cpp is the mature fallback and the natural home for a CLI benchmark harness. MLX Whisper is a Python-based power mode for batch jobs — great for long recordings, wrong for the packaged app.

Default Plus Adapters

Notice the shape: one default plus several adapters. You don't pick a single engine forever; you pick a default for the common case and keep the others swappable behind a common interface. The engine returns text and the language it detected, so the rest of the app never has to care which of the four produced the words. That interface is the whole subject of the next track — here, just internalize that "which STT engine" is a runtime choice, not a permanent marriage.

On-device first, then optimize. The privacy/offline promise decides the category (local engines only); speed, language handling, and packaging decide the default within that category. Never let a faster cloud option reopen the settled question — the audio staying on the machine is the product, not a tunable.

Keep a benchmark harness from day one. With four candidate engines and several models each, you can't reason about 'which is faster/better' from vibes. A tiny CLI that runs the same recorded clips through each engine and prints transcript + latency turns engine choice into data. Firekeeper's has one; yours should too.

Code

One interface, many ears — the engine returns text AND language·swift

struct STTResult {
    let text: String
    let detectedLanguage: String   // "ko", "en", ... — the app routes on this
}

protocol STTEngine {
    var id: String { get }
    func prewarm() async            // load/compile before the first hotkey
    func finalize(_ clip: AudioClip, language: LanguageMode) async throws -> STTResult
}

// WhisperKitEngine, AppleSpeechEngine, WhisperCppEngine all conform.
// The DictationController holds `any STTEngine` and never names a brand.

Exercise

For a voice feature you might build, write the one-sentence rule that decides your engine category before you compare any specific engine. Then list two properties you'd measure to pick the default within that category. Why should the category rule be non-negotiable and the default be swappable?

Hint

The category rule encodes a product promise ('audio stays local'), so it can't bend without breaking the product. The default is an optimization ('WhisperKit handles mixed language fastest for me today'), so it should stay swappable as models and OS APIs improve.

The Non-Negotiable: On-Device

The Four Engines

Default Plus Adapters

Code

External links

Exercise

Progress

Comments 0