C.W.K.
Stream
Lesson 04 of 04 · published

The Gathering Problem

~11 min · gathering-problem, corpus, single-source-of-truth, setup

Level 0Unlit Wick
0 XP0/33 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"Everything I've ever written is somewhere. 'Somewhere' is the problem." — Dad

The Scattered Decade

Here is the real, unglamorous problem Lantern was built for. A person writes for ten years. The writing doesn't accumulate in one tidy place — it lands wherever the writing happened: a guides folder, three book projects, a pile of published essays, a vault of market theses, a manuscript in its own directory, a website's content tree. Six, eight, a dozen homes, each with its own structure, each perfectly reasonable on its own. Together they form a body of work that is real, valuable, and effectively unsearchable as a whole.

The Obvious Answer Is a Trap

The instinct is immediate and wrong: "gather them into one folder." Copy everything into ~/all-my-writing/ and point a search tool at that. It feels like progress. It is a trap with three teeth:

  • It breaks every reference. Links, imports, and build tools that pointed at the original locations now point at nothing.
  • It creates a second copy that drifts. You edit the original; the gathered copy goes stale. Now you have two truths and no way to know which is current.
  • It reorganizes living repos to serve an index. The tail wags the dog: working projects get bent out of shape so a search tool can be lazy.

Moving files to make them findable is like tearing pages out of your books to put them all on one shelf. Now they're together, and now they're ruined.

The move is the trap, not the gathering. Gathering is the right goal. Moving is the wrong mechanism. The whole design of Lantern turns on separating those two — you can gather without moving a single byte, and the next track is entirely about how.

The Real Answer, in One Sentence

Lantern's answer to the gathering problem: the originals stay exactly where they live, and a registry plus a preservation store becomes the one place. You declare where your writing already is; the engine walks those places, reads them, indexes them, and never touches them. The files never know it happened. The gathering is real; the disruption is zero. That's the whole bet — and the rest of this quest is what it takes to make that bet pay.

Code

Why "just copy it all into one folder" quietly ruins the corpus·bash
# The trap, made concrete. "Gather" by copying everything into one folder:
mkdir ~/all-my-writing
cp -r ~/guides/* ~/books/* ~/vault/publish/* ~/all-my-writing/

# What just happened:
#  1. Relative links inside those docs now resolve to the wrong places.
#  2. You now have TWO copies. Edit the original tomorrow and this one lies.
#  3. Your build tools still point at the originals — so which is "real"?
#
# The files are "together" and every one of them is now a liability.
# Lantern never does this. It reads the originals in place and moves nothing.

External links

Exercise

List every place your own writing, notes, or code currently lives — every folder, app, and cloud service. Count them. Now imagine making that whole body searchable as one. Write down what would break if you copied it all into a single folder. Then imagine the alternative: a list that just points at each place. Which one can you update in five years without it rotting?
Hint
The pointer-list survives because it holds no content — only addresses. The copied folder rots because the moment you edit an original, the copy becomes a confident lie.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.