RAG Is Not an Architecture

What RAG actually is

Retrieval-Augmented Generation is a system design pattern, not a model architecture. When a query arrives, the application code retrieves relevant documents from a vector store (or any other source — databases, web search, structured queries) and pastes the retrieved text into the model's context window. The model's weights are unchanged. The model sees the retrieved documents as part of its prompt and generates a response conditioned on them.

RAG is search + paste + prompt

Strip away the names and RAG is three steps: (1) search a corpus for relevant chunks, (2) concatenate them into the model's input, (3) prompt the model to answer using that context. Every step is application code. The model is a black-box generator, identical to one without RAG.

Why people confuse RAG for architecture

Phrases like "the model has RAG capabilities", "built-in retrieval", "memory-enabled AI" all imply something is changing inside the model. Almost nothing is. The same base model can be wrapped in zero or many RAG systems with no weight changes. Two different RAG systems built on the same model can produce wildly different behavior — and that variation lives in the application layer, not in the model.

What changes when you add RAG

Inputs: the model sees more context per query.
Latency: retrieval adds a step before generation.
Cost: larger context = more input tokens = more billing.
Behavior: the model is conditioned on retrieved facts, so it can answer questions about content not in its training data.

What does not change

The model's weights. The model's training. The model's architecture. Any model can be plugged into a RAG pipeline; the pipeline is what gives the system its retrieval ability, not the model.

Code

RAG in 4 lines (the model is unchanged)·python

def rag_answer(query, vector_db, llm, top_k=5):
    docs    = vector_db.search(query, top_k=top_k)
    context = "\n\n".join(docs)
    prompt  = f"Context:\n{context}\n\nQuestion: {query}"
    return llm.generate(prompt)
# llm.generate is the SAME call you'd make without RAG.
# RAG lives in the four lines around it, not inside it.

Exercise

Take any RAG-claimed product (a chatbot, a code assistant, a search tool). Identify what part of the response would change if you swapped the underlying LLM for a different one of similar capability. The retrieval part stays; the generation style changes. That mental separation IS the architecture-vs-product distinction in real life.