Long Conversations — Compaction Strategies

~18 min · conversation, compaction, long-context

Level 0Apprentice

0 XP0/100 lessons0/14 achievements

0/120 XP to next level120 XP to go0% complete

The conversation outgrows its context

Even with 1M-token windows, very long conversations eventually feel "full" — attention degrades, cost climbs, latency suffers. Compaction is the deliberate process of replacing old turns with summaries to keep the active context lean.

Three compaction strategies

Sliding window — keep the last N turns verbatim; drop earlier ones. Cheap; loses context.
Summary buffer — replace older turns with a running summary. Keeps gist; loses verbatim.
Hybrid — pinned system + summary of older + last N verbatim turns. The default for serious systems.

What to keep verbatim

The current task and its intermediate results.
Tool calls in the active loop.
Anything the user just referred to ("as you said earlier").

What to summarize

Resolved sub-tasks.
Background context that won't be quoted again.
Tool results no longer relevant.

Code

Hybrid compaction·python

def compact(messages, system, keep_recent: int = 6, summarize_older=True):
    if len(messages) <= keep_recent:
        return messages
    older, recent = messages[:-keep_recent], messages[-keep_recent:]
    if summarize_older and older:
        summary = summarize_messages(older)
        return [{"role": "user", "content": f"[earlier conversation summary]\n{summary}"}] + recent
    return recent

External links

Anthropic — Long-running conversations

Exercise

On a long conversation log, implement hybrid compaction at turn 30. Compare model performance on a follow-up question with and without compaction. Note any degradation.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousPersona Drift — Catching It Across Turns Next →Re-Anchoring at Each Turn

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.