Weights Are Not Source

"A gigabyte of model weights in your git history is a mistake you commit exactly once."

Weights Live Outside Git

An on-device STT model is hundreds of megabytes to gigabytes. It is data, not source, and it must not live in your repo — commit it once and every clone, every backup, every history walk drags that weight forever. Firekeeper keeps model weights in a cache directory outside the project (under a ~/cwk-models/whisper/ tree), downloaded on first use, never tracked by git. The repo holds the code that manages the model; the model itself is a runtime asset.

The First-Run Tax

The first time a Core ML Whisper model runs, it compiles for the specific machine — a one-time cost of several seconds that looks exactly like a freeze if you don't handle it. Two habits fix this. First, prewarm: kick off model load/compile at app launch, not at the first hotkey, so the model is hot by the time the user actually dictates. Second, show progress: model download and first-run compile both need visible, honest progress (whole-percent download, a 'preparing model' state) so the app never looks hung.

Launch          -> prewarm(): start model load/compile in the background
First dictation -> model already hot -> low latency

Missing model   -> download with whole-percent progress in the overlay
First compile   -> 'preparing model...' state, not a frozen UI

Migrating a Bad Default

One more real detail: Firekeeper once defaulted to a tiny model, which was fast but weak. When the default changed to large-v3-turbo, existing users had the old choice saved in settings. The migration is surgical: bump the old default up once, but keep any model the user deliberately chose. "Upgrade the stale default, respect the explicit choice" is the rule — a blanket overwrite would stomp on a power user's intentional selection.

Ship the code, fetch the weights. Source belongs in git; multi-hundred-megabyte model files belong in a runtime cache the app downloads and manages. Mixing them bloats the repo permanently and couples your version history to artifacts that should be swappable. Keep the boundary clean from the first commit — it's nearly impossible to fix later.

A first-run compile with no UI reads as a crash. Several silent seconds while Core ML compiles is indistinguishable from a hang to a user, who will force-quit. Prewarm at launch and show an honest 'preparing' state; the difference between 'slow' and 'broken' is entirely whether you told the user what's happening.

Code

Prewarm at launch; migrate a stale default without stomping choices·swift

// 1) Prewarm so the first hotkey isn't the first compile.
func applicationDidFinishLaunching() {
    Task { await stt.prewarm() }   // load + first-run compile in background
}

// 2) Migrate the old 'tiny' default up ONCE, keep deliberate choices.
func migrate(_ settings: inout Settings) {
    if settings.sttModel == .tinyLegacyDefault, !settings.userPickedModel {
        settings.sttModel = .largeV3Turbo   // upgrade the stale DEFAULT
    }
    // A model the user explicitly chose is never overwritten.
}

Exercise

For any app that ships a downloaded model, list what belongs in the repo and what belongs in a runtime cache. Then describe the exact user experience of a first-run model compile with (a) no progress UI and (b) a prewarm + progress state. Which one generates a 'the app is broken' bug report?

Hint

Repo: the code that downloads, caches, prewarms, and runs the model. Cache: the weights themselves. Without progress UI, a multi-second silent compile is reported as a freeze or crash; with prewarm and an honest 'preparing' state, the same seconds read as normal startup. The bytes are identical — only the communication differs.

Weights Live Outside Git

The First-Run Tax

Migrating a Bad Default

Code

External links

Exercise

Progress

Comments 0