The Retrieval Contract

~20 min · rag, design

Level 0Scout

0 XP0/41 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

What retrieval owes the generator

RAG works when retrieval keeps a clear contract with the generator: here are the chunks, here is where each came from, here is a confidence signal. Skip any of those and the LLM either hallucinates or refuses to cite.

Three signals every retrieval result should carry

Text — the chunk content, exactly as embedded.
Source — file path, URL, doc id, chunk index. Anything that lets the generator (and the user) trace back.
Score — similarity or rerank score so you can drop low-confidence hits before they reach the prompt.

The minimum-viable RAG loop

question -> embed -> retrieve top-k -> filter by score -> stuff into prompt -> generate -> render with citations

Everything else (query rewriting, multi-query, reranking, summarization) is an optimization. Get the loop running first; optimize what the eval set says is broken.

Code

MVP RAG loop in 30 lines·python

def rag(question: str, k: int = 5, min_score: float = 0.3) -> dict:
    candidates = retrieve(question, k=k * 4)
    candidates = [c for c in candidates if c['score'] >= min_score]
    candidates = candidates[:k]

    if not candidates:
        return {'answer': "I don't have enough information to answer that.",
                'citations': []}

    context = '\n\n'.join(
        f'[{i+1}] (source: {c["source"]})\n{c["text"]}'
        for i, c in enumerate(candidates)
    )
    system = (
        'Answer using ONLY the provided context. Cite sources as [1], [2], etc. '
        'If the context is insufficient, say so explicitly.'
    )
    prompt = f'Context:\n{context}\n\nQuestion: {question}'
    answer = llm.complete(prompt, system=system)
    return {'answer': answer, 'citations': candidates}

External links

Exercise

Implement the MVP RAG loop above against your own corpus. Run 10 questions. Measure: how often the answer cites a real source, how often the answer hallucinates, how often the system says 'I don't know' correctly. Pick the failure mode that hurts most and fix it next.

Progress

Progress is local-only — sign in to sync across devices.

← 🔀 Hybrid Search and RerankingQuiz · 4 questions Next →Prompt Assembly: Stuff, Map-Reduce, Refine

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.