Search Is Not Complete

"They both take text and return text from the corpus. So they must be the same thing, right? No. They're on opposite sides of a latency cliff."

Two Things That Look Like One

Lantern answers two kinds of question that seem like siblings. Search: "find me the passages about long-term investing." Completion: "I'm typing 'the only question a long-term investor' — finish it the way my corpus would." Both take some text, both return text from the same corpus. The lazy instinct is to build one endpoint that does both. That instinct is wrong, and the reason is physics.

The Latency Budget Draws the Line

Search is interactive: a human typed a query and pressed enter, and they'll wait a beat for a good answer. That beat — a few hundred milliseconds — is enough time to consult the keyword index, run a vector search, fuse the two, and optionally rerank. Completion is keystroke-grade: it fires as you type, on every character, and if it can't answer in tens of milliseconds it's worse than useless — it lags behind your fingers. There is no budget for a network round-trip, let alone a model call.

The Job Draws It Too

They're also answering different questions. Search wants relevance: the best matching passages, ranked, wherever they are in the corpus. Completion wants continuation: given the exact prefix under your cursor, what words actually follow it in your own writing? One is a librarian fetching the right books; the other is your own voice, finishing your own sentence. Optimizing one path for both jobs makes it mediocre at each.

When two features share an interface but not a latency budget, keep them separate. Superficial similarity — same inputs, same outputs — is a trap. If one must answer in milliseconds and the other can spend a beat, they will pull the shared implementation in opposite directions. Two clean endpoints that each do one job well beat one clever endpoint that compromises both.

One Engine, Two Endpoints

The two regimes share everything below the surface — the same corpus, the same index, the same chunks — but they expose two separate endpoints with two separate contracts. Search may reach across the network for vectors and reranking, and it announces when it degrades. Completion never leaves the local machine. Keeping them as distinct paths isn't duplication; it's the honest acknowledgment that a keystroke and a query are not the same event, and shouldn't be served by pretending they are.

Code

Two endpoints because a keystroke and a query aren't the same event·python

# Same corpus underneath, two endpoints with two very different contracts.

@app.post("/api/search")
def search(query: str, mode: str = "hybrid", k: int = 8, rerank: bool = False):
    # Budget: ~100s of ms. May use vectors + fusion + optional rerank.
    # Announces degradation. This is the interactive 'find relevant passages' path.
    ...

@app.post("/api/complete")
def complete(prefix: str, k: int = 5):
    # Budget: tens of ms, on EVERY keystroke. No embeddings. No network. No model.
    # This is the 'finish my sentence from the corpus' path. Local index only.
    ...

# Merging these into one endpoint would force completion to carry search's
# machinery — and the keystroke path can't afford a single network hop.

Exercise

Think of two features in an app you use that share an interface but feel different in speed — say, a search box versus its autocomplete dropdown. Estimate the latency budget of each. Would you implement them as one code path or two? List what the fast one would be forced to give up if it shared machinery with the slow one.

Hint

The tell is the latency cliff: if one feature can spend a visible beat and the other must feel instant, a shared implementation forces the fast one to carry the slow one's weight. Two paths that each own their budget almost always win.

Two Things That Look Like One

The Latency Budget Draws the Line

The Job Draws It Too

One Engine, Two Endpoints

Code

External links

Exercise

Progress

Comments 0