Not a RAG Chat

"'Just add a chat box' is how a clean engine grows a second brain and stops being an engine."

The Feature Everyone Eventually Asks For

You build a corpus engine that can find anything you ever wrote. Within a week someone says: "this is great — now let me just chat with it. Ask it questions, get answers." The request is completely reasonable. The naive implementation is a disaster: bolt a language model onto the search endpoint, keep some conversation state, stream answers. You've just grown a brain inside your engine.

Why That Kills the Engine

A chat surface needs things an engine deliberately doesn't have: memory (what did we say three turns ago?), conversation state, identity (who is answering?), and judgment (what does this mean?). The moment the engine sprouts those, it stops being a small sharp tool and becomes a half-built assistant — one that now competes with the actual brain for who owns memory and identity. Do this to each engine in a family and you get eight arguing half-assistants instead of one brain and eight clean tools.

The Right Shape: A Client That Binds Up

The feature is legitimate; it just belongs somewhere else. A "talk to my corpus" surface is a client. It binds to the one brain — the system that already owns conversation, memory, and identity — and it calls Lantern as a retrieval tool. The brain does the judging with Lantern's citable slices in hand; Lantern keeps doing the one thing it's good at. The chat lives above the API, in the brain's world, not inside the engine.

A feature request names a need, not a location. "Let me chat with my data" is a real need. It does not mean "put a chatbot inside the retrieval engine." Separate the need from the placement: satisfy it in the layer that already owns the relevant concerns, and let each component keep its single responsibility. The wrong placement is how responsibilities blur and systems rot.

The Boundary, Stated Plainly

Inside Lantern: no chat, no memory, no soul, no model in the path. If Lantern ever grows a conversational surface, that surface is a companion bound to the brain's conversation — never a second Pippa, never its own identity. The boundary isn't a limitation to route around; it's the thing that lets you trust what the engine is. An engine you can describe in one sentence is an engine you can reason about. Blur it and you lose that forever.

Code

Chat is a client of the brain, not a feature of the engine·python

# WRONG: the engine grows a brain. Memory, identity, judgment leak inward.
@app.post("/api/search")
def search_and_chat(q: str, history: list):        # <-- history? in a search engine?
    hits = retrieve(q)
    return model.chat(history + [prompt(q, hits)])  # <-- engine now owns conversation

# RIGHT: the engine only finds. Chat is a CLIENT that binds to the brain.
@app.post("/api/search")
def search(q: str) -> list[Result]:
    return retrieve(q)                              # citable slices, no model, no memory

# ...elsewhere, in the brain's world, a client composes the two:
def corpus_chat(question, conversation):           # brain owns memory + identity
    slices = lantern.search(question)              # Lantern is just a tool call
    return conversation.respond_with(slices)       # judgment happens up here

Exercise

Think of a clean, single-purpose tool you like (a search box, a converter, a formatter). Imagine the most tempting 'wouldn't it be nice if it also…' feature for it. Now decide: does that feature belong inside the tool, or in a client that composes the tool with something else? Write the one-sentence description of the tool that the feature would have blurred — and defend it.

Hint

If the new feature would force the tool to remember things, know who's using it, or have opinions, it belongs in a client, not the tool. The test is whether you can still describe the tool in one sentence afterward.

The Feature Everyone Eventually Asks For

Why That Kills the Engine

The Right Shape: A Client That Binds Up

The Boundary, Stated Plainly

Code

External links

Exercise

Progress

Comments 0