Things that drift
- Ollama daemon — major versions add features (OpenAI compat, MLX backend on Apple) and occasionally change defaults.
- Model tags —
qwen2.5:7bmay resolve to a different actual GGUF a month from now. - Capabilities — a model that didn't support tools might gain them via a new release.
- System prompt templates — minor template changes are why a Modelfile that worked yesterday produces garbage today.
Update strategy
- Update Ollama itself on a known cadence (monthly) — read release notes for the breaking changes.
- Pin model tags for production. Use
ollama pull qwen2.5:7b-instruct-q5_K_Mnotqwen2.5. The fully-qualified tag is much less likely to silently move. - Re-run your benchmark suite after a daemon update. Catch performance regressions before they ship.
- Keep a "last-known-good" model on disk. If a new pull breaks your prompts, fall back to the previous tag.