Memory — When the Model Should Remember and When It Shouldn't

~16 min · conversation, memory

Level 0Apprentice

0 XP0/100 lessons0/14 achievements

0/120 XP to next level120 XP to go0% complete

Three meanings of "memory"

In-conversation — message history within one session. The model sees it on every turn until it's compacted out.
Cross-conversation — persistent facts the user wants the model to recall across sessions. Implemented in code, stored in a database or vault.
Native memory — provider-managed memory features (OpenAI memory tool, Anthropic memory beta, ChatGPT-style implicit recall).

What to remember

User preferences that persist (timezone, language, formatting choices).
Stable facts about the user's project or domain.
Decisions made in earlier sessions that shape later ones.

What not to remember

Ephemeral task state (current step in a workflow). Use scoped session state.
Sensitive data the user wouldn't want recalled later (payment, health).
Conjectures and one-off speculations.

Who decides

Default to: the user explicitly opts in, or your system explicitly classifies a fact as durable. Implicit memory of everything is a privacy bug, not a feature.

Code

Explicit memory write API·python

@tool
def remember(fact: str, scope: Literal["user", "project"]):
    """Persist a stable fact for future conversations. Only call when the user explicitly asks to be remembered or when the fact is structurally invariant (timezone, role, persistent preference)."""
    memory_store.add(user_id, fact, scope)
    return {"status": "saved", "fact": fact}

External links

Exercise

Audit a conversational system: list every fact persisted across sessions. For each, classify as user-requested, structurally-invariant, or ambient. Remove the ambient class.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousSystem / Developer / User / Assistant Prefill Next →Persona Drift — Catching It Across Turns

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.