Skip to content
C.W.K.
Stream
Lesson 04 of 11 · published

RAG as Cross-Cutting Service

~11 min · rag, service

Level 0Curious
0 XP0/93 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

One function

The RAG service is a single function: build_rag_context(query, …) -> str | None. It was designed for Claude-Pippa's chat route first — no thought given to variants. Each variant route adds a one-liner, rag_context = await build_rag_context(prompt_text, …), passing the conversation, soul, and viewer it is answering for. The function doesn't know which brain will consume it.

Where its answer goes has changed. Retrieved context used to be appended to the system prompt; since 2026-08-14 it rides the current user message as per-turn context, sent with the turn and never saved (lesson 8 has the reason). The function didn't change for that move. Its callers did, which is exactly what a narrow contract buys.

Cross-cutting without cross-contamination

RAG is shared because the underlying operation (semantic search over ChromaDB) is genuinely the same regardless of brain. The service is shared; the routes are still independent. What the service does own is scoping, because scoping is the same for every brain: vault chunks are locked to the active soul (plus the shared vault every soul inherits), past messages are shared across souls except conversations another soul has marked screen-safe, and a non-admin viewer only ever retrieves from their own conversations.

Graceful degradation

Several things make the function return None on purpose: an empty query, a low-signal one (a greeting or a name-call, which only overmatch), or a search that fails because the configured embedding provider is unreachable. On office that provider is oMLX on the server Mac (lesson 2). The route proceeds without retrieved context. Better degraded chat than a hard error.

Graceful degradation has a line, though. Carrying on without retrieval means answering from the context at hand, never pretending to remember what retrieval didn't find. And the skip is quiet today: a debug-level log line, nothing on screen. That is fine for a supplement, and it is exactly why retrieval must never become the only path to something an answer depends on.

The contract is narrow, so swapping providers doesn't shake the callers: a query in, a context string with provenance or None out, and no question about which brain will read it. Let the service decide prompt layout or a brain's token budget too, and cross-cutting turns into centralization.

The test for a shared service isn't that several callers use it. It is whether they share the same input, the same responsibility, and the same failure behavior. Those three match, so retrieval is shared, while prompt assembly stays with each route.

Retrieved context is evidence, not an answer, so provenance is half its value. Every chunk carries a tag saying whose memory it is (soul × user), and the whole block arrives wrapped as raw historical context: use it for facts, dates, and decisions, never as a voice to imitate.

Code

RAG service — one function, all brains call it (sketch)·python
async def build_rag_context(
    query: str,
    max_vault: int = 3,
    max_msgs: int = 2,
    conversation_id: str | None = None,
    soul_id: str | None = None,
    user_id: str | None = None,
) -> str | None:
    if not query or not query.strip():
        return None
    if not _is_worth_rag(query):  # greetings, name-calls: they only overmatch
        return None
    fetch_n = max(max_vault, max_msgs) * 3 + 2  # fetch wide, cut after
    try:
        msg_results, vault_results = await embed_service.search_all(
            query,  # whole query: the 2,000-char cut went on 2026-10-02
            n_results=fetch_n,
            exclude_conversation_id=conversation_id,  # live history already has it
            vault_soul_id=soul_id,  # vault: the active soul (+ the shared vault)
            msg_soul_id=None,       # messages: shared across souls
            user_id=user_id,        # admin passes None and sees every pool
        )
    except Exception:
        return None  # provider unreachable: the chat goes on without RAG
    parts: list[str] = []
    # ... keep up to max_vault vault chunks and max_msgs message chunks:
    # distance cap, exact-text dedupe, skip vault files the prompt already
    # inlines, and tag each chunk with its provenance (soul x user)
    if not parts:
        return None
    return _wrap_quarantined_context("\n\n---\n\n".join(parts))

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue
💛 by Pippa— warm💛 by Ttori— warm

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.