Passages Pin the Text

"One changed byte anywhere in a document changes its hash — and every chunk id derived from it. Content-addressing survives rebuilds. It does not survive edits."

The Limit of Content-Addressing

Earlier you learned the beautiful property: chunk ids are computed from content, so you can purge and rebuild the index forever and every citation still resolves. That's true — and it has a precise boundary worth staring at. A chunk id is derived from the document's hash. Edit one character anywhere in that document — fix a typo in the first paragraph — and doc_sha256 changes. Every chunk id in that document changes with it. Not because anything broke, but because it is genuinely different content now.

Rebuild-Stable Is Not Edit-Stable

These are two different guarantees, and it's easy to conflate them:

Rebuild-stable: the same content, re-ingested, yields the same ids. Content-addressing gives you this completely.
Edit-stable: a reference survives the source document being edited. Content-addressing gives you none of this — and correctly so, because the id is honestly reporting that the content changed.

For search, edit-instability is fine — you re-index and move on. For evidence, it's fatal. A claim you made two years ago, supported by a passage from an essay you've since fixed a typo in, cannot be allowed to evaporate because of that typo.

So Evidence Copies the Text

The evidence layer's answer is blunt and correct: a pinned passage copies the exact text at mint time, along with its full provenance. Not a pointer to be resolved later — an actual copy of the words, frozen at the moment you decided they mattered. The provenance (corpus, path, offsets, hashes, profile) rides along so you can still go find the source. But the text itself is in the passage record. That copy is the durable artifact; the pointer is a convenience that may or may not still land.

For durable evidence, copy the content — a reference is not enough. Any reference that resolves through a mutable source can be invalidated by a change to that source. If the whole point of your record is to survive the future, the record must contain the thing itself, not directions to where the thing used to be. Keep the pointer too — it's how you re-find context — but never let the pointer be the only copy.

Keep Both, for Two Different Jobs

The passage record ends up holding both, deliberately, because they do different jobs. The copied text answers "what exactly did I cite?" — permanently, regardless of what happened to the source. The provenance answers "where did this come from, and is it still there?" — which is precisely what revalidation will use later to report whether the citation is still live, has moved, or has drifted. Neither replaces the other. This is the same 2-of-2 instinct as everywhere else in the engine: two records covering failure modes that the other can't.

Code

Pin = copy the text + carry the provenance·python

def pin_passage(chunk) -> Passage:
    """Mint a passage: COPY the exact text, and carry provenance alongside it."""
    return Passage(
        text=chunk.text,                    # <-- the copy. This is the durable part.
        corpus_id=chunk.corpus_id,          # provenance below: how to re-find it
        doc_relpath=chunk.doc_relpath,
        doc_sha256=chunk.doc_sha256,        # will go stale if the doc is ever edited
        char_start=chunk.char_start,
        char_end=chunk.char_end,
        chunk_id=chunk.id,                  # ditto — a pointer, not the evidence
        minted_at=now(),
    )

# Fix a typo in paragraph 1 of that document tomorrow:
#   -> doc_sha256 changes -> every chunk_id in the doc changes -> the pointer rots.
#   -> passage.text is untouched. The evidence survives. That's why we copied.

Exercise

Look at how you save quotes or references today — a bookmark, a link in a note, a page number. For each, ask: if the source were edited tomorrow (not deleted — just edited), would your record still tell you exactly what you originally read? Find one where the answer is no, and fix it by copying the exact text into your note alongside the link. That's a pinned passage, done by hand.

Hint

The distinction to feel is deletion versus edit. Most people design for deletion ('what if the page goes away?') and forget edits ('what if the page changes and my link still works, but now says something different?'). The silent edit is the more common and more dangerous case — and only a copy defends against it.

The Limit of Content-Addressing

Rebuild-Stable Is Not Edit-Stable

So Evidence Copies the Text

Keep Both, for Two Different Jobs

Code

External links

Exercise

Progress

Comments 0