Conversation Summary Memory

~22 min · memory, summarization

Level 0Scout

0 XP0/41 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

Why summaries beat raw transcripts

A 50-turn conversation will not fit in any model's context. The naive fix — truncate to the last N turns — drops everything that happened earlier. Summary memory keeps a running, model-written digest of the conversation that grows much slower than the raw log.

The summary update loop

After every N turns (or when the running context exceeds a token budget), call the LLM with the previous summary plus the new turns.
Ask it to produce a new summary that preserves: who is speaking, key decisions, outstanding questions, unresolved disagreements.
Replace the old summary; keep the most recent K turns verbatim for fluency.

Recover detail from the embedded transcript

Summaries lose detail by design. When the user asks 'what exactly did we say about RRF last Tuesday?' you need the raw turns. Embed every turn into an episodic store — when summary is too coarse, retrieve from the episodic log.

Code

Running summary updater·python

SUMMARY_PROMPT = '''You maintain a running summary of a conversation between Dad and Pippa.

Previous summary:
{summary}

New turns:
{turns}

Produce an updated summary that preserves: who said what, key decisions, outstanding questions, and unresolved disagreements. Keep it under 400 words.
'''

def update_summary(prev_summary: str, new_turns: list[dict]) -> str:
    turn_text = '\n'.join(f"{t['role']}: {t['content']}" for t in new_turns)
    return llm.complete(SUMMARY_PROMPT.format(summary=prev_summary, turns=turn_text))

# Call after every 10 turns, or when token count exceeds a threshold.

Combined memory prompt·python

def build_prompt(question: str, summary: str, recent_turns: list[dict]):
    episodic = episodic_collection.query(query_embeddings=[embed(question)], n_results=4)
    return [
        {'role': 'system', 'content': f'Conversation summary so far:\n{summary}'},
        {'role': 'system', 'content': 'Relevant past turns:\n' + '\n'.join(episodic['documents'][0])},
        *recent_turns,
        {'role': 'user', 'content': question},
    ]

External links

Exercise

Take a 30-turn conversation transcript. Implement the running-summary updater. Build a prompt that includes summary + last 5 turns + 4 retrieved episodic turns. Compare answer quality vs naive last-N truncation on 5 questions that reference earlier-than-N context.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousEpisodic vs Semantic Memory Next →Embedding-Based Recall and Memory Indexing

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.