Prompt Assembly: Stuff, Map-Reduce, Refine

~22 min · rag, prompts

Level 0Scout

0 XP0/41 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

Three patterns, three contexts where each wins

1. Stuff (default)

Put all retrieved chunks in one prompt and let the LLM answer. Simple, cheap, and the right answer when your top-k chunks fit comfortably under the context window. With 200K-token Claude context, this covers most apps.

2. Map-reduce

For each chunk, generate an intermediate answer (map). Then summarize the intermediate answers into a final response (reduce). Use when the corpus is too big to stuff or when answers must come from synthesizing many sources.

3. Refine

Generate an initial answer from the first chunk, then iteratively update it with each subsequent chunk. Useful for ordered material (transcripts, time-series) where new context should refine rather than override.

How to pick

Default to stuff. Switch to map-reduce when context overflows or when the eval shows the model under-uses chunks past position 5. Switch to refine for ordered content. Almost no app needs all three at once.

Code

Map-reduce sketch·python

def rag_map_reduce(question: str, k: int = 20):
    chunks = retrieve(question, k=k)
    intermediates = [
        llm.complete(f'Answer this question using ONLY the chunk below.\n\nQuestion: {question}\n\nChunk: {c["text"]}')
        for c in chunks
    ]
    combined = '\n\n'.join(intermediates)
    final = llm.complete(
        f'Combine these answers into a single coherent response. Discard contradictory or irrelevant pieces.\n\n{combined}'
    )
    return final

External links

Exercise

On 10 questions that need synthesis across 5+ chunks, compare 'stuff' vs 'map-reduce'. Track answer quality and total token cost. Decide for which question shapes the cost is worth it — and write that decision into a comment in your retriever.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousThe Retrieval Contract Next →Citations and Provenance That Hold Up

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.