C.W.K.
Stream
Lesson 02 of 04 · published

The NAS Owns the Binaries

~11 min · references, storage-ownership, tombstone, no-hoarding

Level 0Empty Shelf
0 XP0/35 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"Own everything you could regenerate. Reference the one thing you couldn't."

The Hoarding Instinct

There's a strong pull, when you build an engine, to make it self-contained: the engine should own everything it needs. Copy the videos into its own storage, and now it's complete, portable, dependent on nothing. Recall refuses, and the refusal is one of its cleanest lines: the storage owns the original media. Recall stores paths, metadata, hashes, and the evidence it derives — it never copies the archive into its own database.

The cost of hoarding stacks up immediately. You've just duplicated thousands of multi-gigabyte files, so now there's a second archive to store, back up, and keep in sync. And that last one is fatal: two copies means a standing question of which copy is real — the exact ambiguity Track 2 refused to accept from a synced folder. You'd have imported that whole bug class in exchange for a feeling of self-sufficiency.

The Inversion Worth Stealing

Look at what Recall actually keeps versus what it points at, because the split is the opposite of the instinct:

  • It owns the evidence, the releases, the segments, the index — all of which it could regenerate if lost (re-transcribe, re-project, re-index).
  • It references the original footage — the one artifact that cannot be regenerated by anyone, ever. If that file is gone, no amount of compute brings it back.

That's the inversion: the engine owns everything it could rebuild and merely references the single thing it couldn't. It sounds backwards until you see the logic — the irreplaceable thing deserves an owner whose entire job is keeping bytes safe, not an application that's also running jobs, mutating schemas, and being restarted by a service manager mid-write. Storage is good at storage. The engine should be good at meaning. Don't make the engine the custodian of the one thing it can't recreate.

Tombstones: The Memory Outlives the File

This split produces a quietly moving consequence. When a scan finds that a video is no longer where it lived, Recall doesn't delete the record — it marks it missing, with the date it went absent. A later scan that finds it again clears the mark. And in the meantime, that tombstoned video stays searchable and exportable. It just can't be queued for work or opened as live source media, because the bytes aren't there.

Sit with what that means for a memory engine. You lose the file, but you keep the memory: the transcript, the timestamps, the summary, what was said and when — all of it survives, because the evidence was derived and owned while the binary was only referenced. You can still search a video you no longer have and read exactly what you said in it. You just can't watch it. That's not a consolation prize; that's the architecture being honest about two different kinds of loss, and refusing to pretend the record and the footage are the same thing.

Code

Reference the bytes; own the meaning·sql
CREATE TABLE videos (
  video_id       TEXT PRIMARY KEY,   -- stable logical identity
  source_id      TEXT NOT NULL,      -- WHICH archive root
  relative_path  TEXT NOT NULL,      -- WHERE it lives (a reference)
  size_bytes     INTEGER,
  modified_ns    INTEGER,
  source_sha256  TEXT,               -- identity of the bytes
  missing_since  TEXT,               -- tombstone: gone since when
  title          TEXT
  -- note what is NOT here: the video bytes.
);

-- Owned (regenerable): runs, releases, segments, fts, summaries
-- Referenced (irreplaceable): the original footage on the storage

-- A tombstoned video stays SEARCHABLE and EXPORTABLE.
-- It just can't be queued or opened as live media.
-- The memory outlives the file.

External links

Exercise

Look at a system you've built that ingests files or external records. Does it copy them in, or reference them? For each thing it copies, ask: could I regenerate this if I lost it? If yes, owning it is fine. If no, ask whether your application is really the right custodian for something irreplaceable — and what your 'which copy is real?' rule is. Then design the tombstone: when a referenced source disappears, what survives, and what stops being possible?
Hint
Two questions: (1) for everything you store, is it regenerable? Own the regenerable freely; be very suspicious of being the sole custodian of anything that isn't. (2) When a referenced thing vanishes, does your system delete the record or mark it absent? Deleting throws away the derived memory along with the file — but those are two separate losses, and only one of them was forced on you.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.