A Model Store Is Not an Inference Server

"Four and a half terabytes of weights the household cannot run, kept on purpose, verified once, byte for byte. The store's job is to know what it has. Running is someone else's job."

What the Store Holds

The household's model store is a small engine whose only verbs are fetch, monitor, verify, land and catalog. Its own house rule says what it is not: "not an inference runtime, model evaluator, quantizer". Read from its catalog on 2026-09-15, it has landed five checkpoints — GLM-5.3 and GLM-5.3-Flash in BF16, DeepSeek-V4.1-Flash in its mixed FP4 and FP8, Kimi K3 as the full repository, Qwen3.8-Flash-Next in BF16 — 4.58 TB in 767 files, on a cold archive the fleet track's backup mount reaches. The largest pool in the house is 0.512 TB. Three of the five cannot be loaded by any Mac the household owns in the form the store holds them, and that is the point: a store keeps the original, and a runtime keeps what runs.

Provenance Is the Product

What a store adds that a download does not is the second column of its catalog: the git revision of the source repository at landing, and a verification that the bytes match it. This quest checked the five landed weight sets blob by blob against each source's current main branch: all five are byte-identical; only small text files — chat templates, a README, evaluation configs, one tokenizer script — had moved since. So the household knows, for each checkpoint, exactly which published weights it has, and it would know if a source quietly replaced them. The inference hub does not need to know this; it needs a quantized runnable and a warm cache. The store does not need to run anything; it needs to be right. Separating the two is the fleet track's single-writer rule applied to weights: one place owns the truth of what a model is, and every runtime is a client of that truth.

Why the Originals, Not the Runnables

A 4-bit MLX checkpoint is derived from a BF16 original by a choice of bits, group size and mode, and the choice can be remade — the quantize lesson does it — while the original cannot be recovered from the derived file. Keeping 1.5 TB of BF16 for a model that will only ever run at 4 bits looks wasteful until the next quantization method arrives, or the next machine with a larger pool, or a question about whether a runtime's numbers changed. The store also holds Kimi K3, which fits no Mac in the house at any bits the quest can name; that is a checkpoint kept for a machine that does not exist yet, which is what a store is for and an inference server never is. The brief this quest was written from got the catalog wrong twice — a missing model, a size attributed to the wrong one — and the engine's own catalog corrected it; the store is the evidence, and this track's arithmetic starts from its bytes.

Code

catalog.py — the five landed checkpoints, as the store's engine reports them·python

#!/usr/bin/env python3
"""A model store is not an inference server. The household's store keeps original
checkpoints in a cold archive and catalogs them; nothing in it runs. The catalog as the
store's own engine reported it on 2026-09-15, with a column no runtime cares about
(the git revision) and one every runtime cares about (bytes)."""
catalog = [  # release, variant, source repo, revision, bytes, files
    ("GLM-5.3",            "BF16",           "zai-org/GLM-5.3-BF16",          "304b8051", 1_506_693_048_081, 291),
    ("GLM-5.3-Flash",      "BF16",           "zai-org/GLM-5.3-Flash-BF16",    "f12e0fe1",   642_676_400_602, 130),
    ("DeepSeek-V4.1-Flash","FP4-FP8-mixed",  "deepseek-ai/DeepSeek-V4.1-Flash","df42c109",  510_313_345_254,  88),
    ("Kimi K3",            "full-repo",      "moonshotai/Kimi-K3",            "2496450e", 1_560_998_983_759, 114),
    ("Qwen3.8-Flash-Next", "BF16",           "Qwen/Qwen3.8-Flash-Next",       "de4b8e4d",   360_023_351_514, 144),
]
total = sum(c[4] for c in catalog)
print(f"{'release':20} {'variant':14} {'source':34} {'rev':9} {'TB':>6} {'files':>5}")
for rel, var, src, rev, b, n in catalog:
    print(f"{rel:20} {var:14} {src:34} {rev:9} {b/1e12:6.2f} {n:5d}")
print(f"\n{len(catalog)} checkpoints, {total/1e12:.2f} TB on the archive; the largest single Mac pool in the house is 0.512 TB")
print("landed weight sets were checked blob-by-blob against each source's main branch: all five byte-identical; only text files had moved")

Exercise

List every model checkpoint on your disks with its source and, if you can find it, the revision you downloaded. Write on your card which ones are originals and which are derived (quantized) forms, and for each derived one whether you still have its original. Then decide which of them your store should keep and which your runtime should.

Hint

If a quantized file has no original anywhere you control, you cannot requantize it, verify it, or know whether the source changed. That is fine for a model you will replace next month and not fine for one you will run for a year.