Video vs Fingerprint

"'Which video is this?', 'did it change?', and 'are these the exact bytes?' are three questions. They need three answers."

Three Questions Hiding Behind 'The File'

When Recall scans thousands of videos, 'is this the same file as before?' turns out to be three different questions, each needing a different identity and — crucially — each with a different cost. Getting them tangled either makes the scan unbearably slow or lets changed videos masquerade as unchanged.

video_id = hash of the source and the relative path. This is the stable logical identity — the slot. The file at 2026/07/talk.mov keeps the same video_id even if you re-export and replace it. It answers 'which video are we talking about?' and is what all the durable rows hang off of.
cheap_fingerprint = hash of the source, path, size, and modified-time. This detects change. It moves the instant the observed bytes change — a new export has a new size or mtime — but it computes without reading the file's contents. It answers 'did this change since I last looked?' cheaply.
source_sha256 = hash of the entire file contents. This is the exact, expensive, ground-truth identity of the bytes. It requires reading every byte, so Recall defers it until a job is actually selected to run. It answers 'are these literally the same bytes?' with certainty.

Why Three, and Not Just the Real Hash?

The obvious purist move is 'just SHA-256 the file — that's the true identity.' On an archive of multi-gigabyte 4K videos, that's a non-starter: hashing every byte of every file on every scan would make inventory take hours and hammer the network filesystem. So Recall tiers the identity by cost. The cheap fingerprint does the constant, high-volume work of change detection during scanning; the expensive full hash is computed only for the handful of videos a job actually touches. You pay the cheap cost always and the expensive cost only when it matters.

This is exactly the pattern behind tools you already use. A sync protocol compares cheap signals (size, timestamp) to decide whether to do the expensive byte-level work. An HTTP cache sends a cheap fingerprint so the server can say 'unchanged' without re-transmitting the whole body. The principle is the same: a cheap identity gates the expensive one. Detect change cheaply and constantly; confirm identity expensively and rarely.

Separating the Slot From Its Contents

The deepest of the three ideas is keeping the slot (video_id) separate from the contents (fingerprint and full hash). Because video_id is stable across edits, all the history for a logical video — its releases, corrections, summaries — stays attached even when the underlying file is re-exported. The fingerprint and full hash then track that the contents beneath that stable slot have moved, so Recall can say 'same video, new bytes' rather than losing the history or pretending nothing changed. One identity holds the thread of history; the others detect that the material on it changed. Keep those jobs in separate hands.

Code

Three identities, three costs, three jobs·python

import hashlib

# 1. Stable logical slot — cheap, constant. History hangs off this.
video_id = sha256(f"{source_id}\x00{relative_path}")
#   survives re-export at the same path -> 'which video is this?'

# 2. Change detector — cheap, no file read. Runs every scan.
cheap_fingerprint = sha256(
    f"{source_id}\x00{relative_path}\x00{size_bytes}\x00{modified_ns}"
)
#   moves when observed bytes move -> 'did it change?'

# 3. Ground-truth bytes — EXPENSIVE, reads whole file.
#    Deferred until a selected job actually runs.
source_sha256 = sha256(open(path, 'rb').read())
#   exact byte identity -> 'are these literally the same bytes?'

# cheap fingerprint gates the expensive full hash.

Exercise

Find something in your own work whose 'true' identity is expensive to compute — a large file, a big dataset, an external record you'd have to fully fetch. Are you computing the expensive identity every time you check for change, or do you have a cheap proxy? Design the two-tier scheme: a cheap change-detector (size, timestamp, version tag) plus the expensive exact identity computed only when the cheap one signals a change or a job actually needs it.

Hint

Look for a place that either re-hashes/re-fetches everything on every pass (too slow) or trusts a timestamp alone as if it were the content (unsafe). The balanced design uses the cheap signal to decide whether to bother, and the expensive exact identity to confirm — and keeps a third, stable identity for the logical slot so history survives the contents changing.