The Model Goes Above

"Put the flaky part where its failure is a shrug, not a crash. Everything else follows from that one placement."

Where Everyone Else Puts the Model

The default RAG shape puts a language model inside the retrieval flow. You ask a question, the system retrieves some chunks, stuffs them into a prompt, and the model writes an answer. The model is load-bearing: without it, there is no result at all. Retrieval and generation are fused into one pipeline, and that pipeline is only as reliable, as fast, and as reproducible as the least reliable, slowest, least reproducible thing in it — the model.

Where Lantern Puts It

Lantern draws the line one level higher. The engine's job ends at finding: it returns citable slices, ranked, with provenance. That's a complete, useful result on its own — you can read the slices, cite them, show them in a UI. Judgment — turning those slices into an answer, deciding what they mean, writing prose — happens above the API, in a separate brain, as a separate step. The model is still in the picture; it's just not in the path.

Why the Model Is the Weak Link

Three properties make a language model the wrong thing to depend on for a foundation:

It's expensive. Every call costs money or GPU time. A search you run on every keystroke can't afford that.
It's non-deterministic. The same query can yield different output. A foundation that gives different answers each time isn't a foundation.
It's sometimes unavailable. Servers go down, rate limits hit, networks drop. If finding depends on the model, then when the model is down, finding is down.

None of those are flaws you can engineer away — they're the nature of the thing. So you place it accordingly.

Depend on your most reliable component, not your most powerful one. The impressive part (the model) is the fragile part. Build your foundation out of the boring, deterministic, always-there parts — indexes, hashes, string matches — and let the powerful-but-flaky part sit on top, where its absence degrades the experience instead of collapsing it.

The Payoff Is a Foundation You Can Trust

Once finding is a pure function of the index and the query, everything downstream gets sturdier. Search runs on every keystroke because it's cheap. Results are reproducible because nothing rolls dice. The corpus is queryable at 3 a.m. when the model server is rebooting. You bought all of that with a single decision about where the model sits — not inside the engine, but above it.

Code

Model in the path (RAG) vs model above the path (Lantern)·python

# RAG default: the model is IN the path. No model, no result.
def answer_rag(question: str) -> str:
    chunks = retrieve(question)                 # find
    return model.generate(prompt(question, chunks))  # <-- fused; model is mandatory

# Lantern shape: the engine stops at finding. The model sits ABOVE, optional.
def search(query: str) -> list[Result]:
    return rank(index.match(query))             # pure fn of index + query; no model

# Judgment is a SEPARATE step, in a separate brain, above the API:
def answer_above(question: str) -> str:
    results = search(question)                   # deterministic, always works
    if brain.available():
        return brain.synthesize(question, results)  # nice-to-have
    return render_citations(results)             # graceful fallback: show the slices

Exercise

Take any system you've built or used that calls an AI model. Draw where the model sits in the flow. Now ask: if the model were unavailable right now, what would still work and what would go completely dark? Everything that goes dark is depending on the model. Sketch how you'd move the model 'above' so the core still functions without it.

Hint

The move is almost always: make the layer below the model return something useful on its own (a list, a match, a slice), so the model becomes an enhancement of a working result rather than the only thing that produces a result.

Where Everyone Else Puts the Model

Where Lantern Puts It

Why the Model Is the Weak Link

The Payoff Is a Foundation You Can Trust

Code

External links

Exercise

Progress

Comments 0