RAG vs Long Context Is a Routing Problem

~26 min · rag, long-context, routing

Level 0Window Watcher

0 XP0/50 lessons0/13 achievements

0/100 XP to next level100 XP to go0% complete

No religion, just fit

RAG is better when you have a large corpus and need selective retrieval, citations, or fast repeated lookup. Long context is better when the task needs cross-reference over a bounded set that fits and should be seen together. Treat them as tools in a routing decision, not as competing philosophies.

Production usually combines both

Retrieve candidate files via RAG, then load the few that actually matter in full into long context. That gives you library scale plus desk-level reasoning. Pure RAG drops cross-references; pure long-context drowns in irrelevant material; the hybrid splits the difference.

Route by task shape

Ask whether the task is search, synthesis, audit, or transformation. Search wants retrieval. Deep synthesis across a few documents wants long context. Transformation wants the exact source files and tests, not a vector-store horoscope. The shape of the task chooses the strategy; ego does not get a vote.

Code

Context router·python

def choose_context_strategy(task):
    if task.corpus_tokens > task.model_window:
        return "RAG"
    if task.requires_exact_cross_reference:
        return "long_context"
    if task.needs_citations_across_many_docs:
        return "RAG_then_load_top_sources"
    if task.is_transformation_of_known_files:
        return "load_exact_files"
    return "small_context_with_pointers"

Routing matrix·text

                       FITS IN WINDOW         BIGGER THAN WINDOW
NEEDS HOLISTIC         long_context           summarize_then_long
NEEDS SELECTIVE        load_exact_files       RAG
NEEDS CITATIONS        long_context+cite      RAG_then_load_top

External links

Gemini long context guide

Exercise

Classify four real tasks as RAG, long context, hybrid, or pointer-only. Explain the routing rule for each.

Hint

Look at corpus size, cross-reference needs, and citation requirements.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousDo Not Put the Whole Library on the Desk Next →Tool Results Need Summaries, Not Dumps

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.