C.W.K.
Stream
Lesson 01 of 04 · published

No Single Flag

~10 min · single-flag, idempotency, staleness, cache-invalidation

Level 0Empty Shelf
0 XP0/35 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"Done according to what? A flag that can't answer that question is a bug waiting for an input to change."

The One-Bit Lie

Almost every pipeline grows a processed = true flag. It's the obvious way to avoid redoing work: mark each item done, skip the done ones next time. And it's fine — until an input changes. Then the flag becomes a quiet lie. The item is marked done, but done relative to what? The source file that has since been re-exported? The audio settings you've since changed? The model config you've since upgraded? A single boolean can't say. It collapses a dozen distinct questions into one bit, and when any of the underlying inputs move, the bit still reads true and the genuinely-changed work is silently skipped.

This is the same trap as a stale cache. A cache entry says 'I have the answer' — but the answer was computed for a specific input, and if the input changed, the cached answer is wrong while still looking authoritative. A done flag is exactly a cache with no key: it remembers that work happened but not what it was done against. Cache invalidation is famously one of the hard problems in computing, and a naked done-flag walks straight into it.

'Already Done' Only Means Something Relative to an Identity

The fix is to stop asking 'is this done?' and start asking 'is this done for this exact identity?' Instead of a boolean on the video, Recall records what was done against precisely-defined identities: this source content, this audio configuration, this provider configuration, this release. Now 'already done' is a real question with a checkable answer: hash the current inputs, look for a record under that exact identity. Found it? Genuinely done, skip safely. Not found? Something changed, do the work. The skip is now correct because it's tied to what the work actually depended on.

The difference is the difference between 'I did something to this video once' and 'I produced this exact output from these exact inputs.' Only the second one is safe to skip on, because only the second one goes false the instant an input changes. The rest of this track is about choosing those identities well — one for each kind of change you need to notice.

Why This Matters More as Systems Grow

On a toy project, a done-flag mostly works, because inputs rarely change under you. On a real archive that gets re-scanned, re-exported, re-configured, and re-run over months, inputs change constantly — and every one of those changes is a chance for a naked done-flag to skip work that needed redoing. The bug doesn't announce itself; it shows up as 'why is this video's transcript from the old model?' long after the cause. Layered identity is the cost you pay up front to never debug that class of silent staleness later. Precision now, or mystery skips forever.

Code

A boolean forgets its key; an identity remembers it·python
# WRONG: a flag that forgets what it was done against
if video.processed:
    skip()   # done... relative to WHAT? source? config? model?
             # any input changed -> still 'true' -> silently skips

# RIGHT: 'done' is checked against the exact identity it depended on
identity = (source_sha256, audio_config_sha, asr_config_sha)
if ledger.has_completed(identity):
    skip()   # provably done for THESE inputs
else:
    do_the_work()   # an input changed -> identity differs -> redo

# A done-flag is a cache with no key. Give it the key.

External links

Exercise

Find a 'processed' / 'done' / 'synced' flag in your own work. List every input whose change should invalidate that flag — the source, the settings, the version of the code that produced it. Now ask: if any of those change, does your flag notice, or does it keep reading true? Redesign completion as a record keyed by the identity of those inputs, and describe the exact lookup that replaces the boolean.
Hint
The test is: name a change that should force a redo, then check whether your current flag catches it. If the flag stays true after a meaningful input changed, it's a keyless cache. Fix it by hashing the inputs that matter into an identity and recording completion under that identity — now 'already done?' answers correctly for every change you enumerated.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.