Citations and Provenance That Hold Up

~22 min · rag, citations

Level 0Scout

0 XP0/41 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

Why citations matter beyond cosmetics

An answer without a citation is unverifiable. Users (and you, debugging) need to click through to the source chunk to confirm the model did not invent details. Citations also act as a forcing function — models hallucinate less when prompted to cite.

Two citation styles

Inline numeric (Wikipedia-style). Easy for the model to produce, easy to render. '... was released in 2024 [3].'
Inline link. Render the source title or URL inline. Heavier visually but clickable.

The citation contract

Tell the model exactly what citation format you expect, and refuse to render an answer that breaks it. A regex that finds [\d+] tokens and confirms each has a matching candidate is enough to catch most hallucinated citations.

Code

Validate citations match the candidate set·python

import re

def validate_citations(answer: str, candidates: list[dict]) -> tuple[bool, list[int]]:
    cited = sorted({int(m.group(1)) for m in re.finditer(r'\[(\d+)\]', answer)})
    valid_range = set(range(1, len(candidates) + 1))
    bad = [c for c in cited if c not in valid_range]
    return (len(bad) == 0, bad)

result = rag(question)
ok, bad = validate_citations(result['answer'], result['citations'])
if not ok:
    print(f'WARN: hallucinated citations {bad}')

Render the answer with clickable sources·python

def render(answer: str, candidates: list[dict]) -> str:
    sources_md = '\n'.join(
        f'[{i+1}] [{c["source"]}]({c["source"]})  — chunk {c["chunk_index"]}'
        for i, c in enumerate(candidates)
    )
    return f'{answer}\n\n---\n\n**Sources**\n{sources_md}'

External links

Exercise

Add the citation validator above to your RAG. Run 30 questions. Log every hallucinated citation. Adjust the system prompt to be more strict about only citing provided sources. Re-run and confirm the hallucination rate drops.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousPrompt Assembly: Stuff, Map-Reduce, Refine Next →When RAG Is the Wrong Answer

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.