Verify After Landing

The moment of truth is after the bytes stop moving

Acquisition ends, storage begins — and the two moments have different failure modes. During transfer, failures are loud (interrupted connections, size mismatches). At rest, failures are silent: disks flip bits without ceremony, files get truncated by interrupted copies, filesystems mark blocks bad and quietly relocate. Nothing announces storage corruption; the file just sits there, wrong, until the day something loads it.

Verification after landing is therefore not a download concern — it is a storage discipline with three moments:

Landing check (once). The full ritual from this track: size, then digest, then — for formats that support it — a structural parse (open the safetensors header, dump the GGUF metadata). The parse catches what a digest cannot describe: a file that is byte-perfect but structurally inconsistent with its claimed format.
Migration check (every copy). Every time bytes move — disk to disk, machine to NAS, cloud to local — is a new chance for corruption. The digest record travels with the artifact precisely so every copy can re-earn its identity. A migration without a re-check is an unverified copy wearing a verified file's history.
Rest check (periodically). A slow sweep of stored digests — monthly, quarterly, whatever matches your scale — catches bit rot and silent decay while recovery is still possible (that is what replication is for).

What each instrument sees

Use all three instruments because they see different things. Size catches truncation cheaply. Digest proves byte-identity with the recorded reference — the workhorse. Structural parse proves format sanity: the JSON header decodes, the offsets land inside the file, the tensor count matches the index. A file can fail any one of these while passing the others: truncated-but-lucky sizes, corrupted-but-parseable headers, intact files with wrong content. Cheap checks in order, expensive trust last — the review desk's rule, applied to your own storage.

Every copy re-earns its identity. A verified original does not vouch for its copies. Each landing — acquisition, migration, or restoration — gets the same size-then-digest-then-parse ritual.

Making it routine

The whole discipline compresses to a script you already wrote in this track's exercises: walk the archive, compare sizes and digests against SHA256SUMS, report a line per file. Schedule it, run it after every migration without exception, and treat any FAILED as a restore event — pull from replication, not optimism. The archive's health is not a state you achieve; it is a measurement you keep taking.

Code

The three-moment verification, one script·bash

cd ~/models/<model>

# Landing / migration / rest — same ritual:
# 1) sizes (instant, catches truncation)
stat -f "%z %N" *.safetensors

# 2) digests (the workhorse)
shasum -a 256 -c SHA256SUMS 2>&1 | grep -v ': OK$' || echo "all digests OK"

# 3) structural parse (format sanity, catches what digests can't describe)
python3 - <<'EOF'
import json, struct, glob, sys
for path in sorted(glob.glob("*.safetensors")):
    with open(path, "rb") as f:
        n = struct.unpack("<Q", f.read(8))[0]
        header = json.loads(f.read(n))
    import os
    tensors = {k: v for k, v in header.items() if k != "__metadata__"}
    print(f"{path}: {len(tensors)} tensors, header {n}B, file {os.path.getsize(path)}B -> parses OK")
EOF

# A FAILED digest at rest = restore event (pull from replication),
# never a shrug and never an overwrite-in-place.

Exercise

Write (or reuse) the three-moment verification script and run it against an archive directory you control. Then simulate two failures on a copy: truncate a file (head -c), and corrupt one byte in another (dd seek). Run the script and confirm each failure is caught by the right instrument. Record which instrument caught which failure.

Hint

The truncation should fail the size check first; the single-byte corruption keeps the size but fails the digest. That difference is exactly why the ritual has both.

The moment of truth is after the bytes stop moving

What each instrument sees

Making it routine

Code

External links

Exercise

Progress

Comments 0