Search With the Brain Down

"The test isn't whether it works when everything is up. The test is what still works when the fancy part is gone."

The 3 A.M. Test

Imagine the embedding-and-rerank server is down — rebooting, out of memory, unplugged, doesn't matter. It's 3 a.m. and you need to find a line you wrote two years ago. In a model-in-the-path system, you're stuck: no model, no search. In Lantern, you type your query and the corpus answers, because the paths you rely on most were built to touch no model at all. That's not luck or a fallback that kicked in — it's the shape of the engine.

Two Paths That Touch No Model

Two of Lantern's retrieval paths are pure functions of the local index and your query:

Keyword search (full-text) matches your terms against an inverted index living right there on disk. No network, no model, just term matching and ranking.
Phrase completion mines the corpus for continuations of what you typed, again from the local index, at keystroke speed.

Neither one has a wire to the model server. You could delete the model server from existence and both would return identical results.

What Degrades, and How

The paths that do need the model server — dense vector search, the vector half of hybrid, and reranking — degrade honestly rather than pretending. A pure vector query returns a clear service unavailable when the model is unreachable. A hybrid query serves its keyword half and flags in the response that the vector half was skipped. Nothing silently returns worse results dressed up as good ones. Degradation is visible, named, and never contagious to the deterministic paths.

Make your foundational path a pure function of local state. If the thing users depend on most can be computed from data you already hold, without any network call, then no outage anywhere else can take it down. Reach out to remote services only for enhancements — and design those reaches so their failure is a labeled degradation, never a collapse.

The Guarantee Is Structural, Not Hopeful

This is worth saying twice: the always-works property isn't caching, isn't a retry loop, isn't a generous timeout. Those are patches that reduce the chance of failure. Lantern's deterministic paths cannot fail from a model outage because they were never connected to the model. Structural guarantees beat hopeful ones — you don't have to test every failure mode of a dependency you don't have.

Code

FTS never depends on the model; hybrid degrades out loud·python

def search(query: str, mode: str = "fts") -> Envelope:
    if mode == "fts":
        # Pure local index. No network. Cannot be taken down by a model outage.
        return Envelope(results=fts_match(query), degraded=[])

    if mode == "vector":
        if not model_server.reachable():
            raise ServiceUnavailable("vector search needs the embedding server (503)")
        return Envelope(results=vector_match(query), degraded=[])

    if mode == "hybrid":
        fts_hits = fts_match(query)               # always available
        try:
            vec_hits = vector_match(query)         # best-effort enhancement
            return Envelope(results=fuse(fts_hits, vec_hits), degraded=[])
        except ServiceUnavailable:
            # Serve the half that works — and SAY the other half is missing.
            return Envelope(results=fts_hits, degraded=["vectors_unavailable"])

Exercise

Write down the single feature of something you use daily that you'd be most upset to lose during an outage. Trace every remote dependency it has. For each, decide: is this reach essential to the feature, or an enhancement that could degrade gracefully? Redesign one essential reach into an enhancement, so the core keeps working when that service is down.

Hint

The recurring move: precompute or index locally what you can, so the feature answers from data you already hold. The remote call then only makes a working answer better, instead of being the difference between an answer and a blank screen.

Search With the Brain Down

The 3 A.M. Test

Two Paths That Touch No Model

What Degrades, and How

The Guarantee Is Structural, Not Hopeful

Code

External links

Exercise

Progress

Comments 0