Variant Trees and Revision Pins

A repository is a tree, not a snapshot

Model hubs borrowed their storage model from git, and that inheritance is more than cosmetic. The URL you bookmark — owner/model — names a repository, and by default every read of it resolves to whatever the main ref currently points at. Authors can push corrections, swap file formats, replace a native release with a quantized one, or empty the repo entirely. Between your download today and your re-download next year, the address may serve two different objects.

The fix is the same one git gives programmers: name the commit. Every state of the repository is addressable by its revision sha — the commit hash the API reports as sha. A URL plus a pinned revision names a frozen, reproducible object; the URL alone names a policy ("whatever is current").

Branches and refs you will meet

main — the default. Moving, editable, the object of silent substitution.
Named branches — conversion branches (a repo keeping FP32 and BF16 variants as refs), legacy snapshots, or staging refs. Useful alternates; always record which ref you read.
PR refs — community proposals not yet merged. Fine for inspection, risky for acquisition: they can be rebased away.
Tags/releases — the closest thing to an authorial "this is a version" statement, when present. Prefer a tag over a bare branch when both exist.

Sharding interleaves with this: a model too large for one file ships as model-00001-of-00003.safetensors and friends plus an index that maps tensor names to shards. The shard count is part of the snapshot's identity — a repo that reshards from 3 to 2 files has changed objects on you even if every tensor survives. Your acquisition record should capture the file list, not just the revision.

Acquire a coordinate, not an address. The full identity of an acquisition is repo + ref + revision sha + file list + digests. Anything less is pointing at a moving object and calling it yours.

Making the pin a habit

The practical ritual is one line at acquisition time: query the metadata endpoint, record the sha next to your copy, and from then on treat your pinned copy as the reference — re-resolving the URL only when you deliberately want "current". When a later query returns a different sha, you have a fact to reason about (did the author improve something? swap a format? fix a license?) instead of a silent substitution under your feet.

Code

Pin, then detect drift·bash

# Acquisition day: record the coordinate
REPO=mistralai/Mistral-7B-Instruct-v0.2
SHA=$(curl -s https://huggingface.co/api/models/$REPO | python3 -c "import json,sys; print(json.load(sys.stdin)['sha'])")
echo "$REPO $SHA $(date -I)" >> ~/models/pins.log

# Later: has the address moved?
NOW=$(curl -s https://huggingface.co/api/models/$REPO | python3 -c "import json,sys; print(json.load(sys.stdin)['sha'])")
grep "$REPO" ~/models/pins.log | tail -1   # what you archived
[ "$NOW" = "$(awk '{print $2}' <<< "$(grep $REPO ~/models/pins.log | tail -1)")" ] \
  && echo "stable" || echo "DRIFTED: main now $NOW"

# Fetch a SPECIFIC frozen revision (the pin), not 'current':
#   resolve endpoint with ?revision=<sha> in supported tooling,
#   e.g. hf download $REPO --revision $SHA

Exercise

Add a pins.log ritual to your archive: for two repositories you hold or plan to hold, record repo, sha, date, and file list today. Then simulate drift: change one character of a recorded sha and verify your drift check catches it. Finish by writing where in your archive layout the pin record lives alongside the bytes.

Hint

The file list belongs with the pin. Upstream, the pin catches re-shards for you; at home, on your own storage, the one-line fingerprint (shard count + total bytes) is what catches a partial restore.

A repository is a tree, not a snapshot

Branches and refs you will meet

Making the pin a habit

Code

External links

Exercise

Progress

Comments 0