C.W.K.
Stream
Lesson 03 of 05 · published

When the Model Disagrees with Itself

~22 min · self-disagreement, consistency, audit

Level 0Window Watcher
0 XP0/50 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete

The same question, different answers

Long sessions sometimes produce contradictory answers to the same question across turns. The model is not lying. It is responding to subtly different context — different recency, different attention sinks, different cache state. Self-disagreement is a debugging gift: it tells you which facts are not robust under context drift.

Surface, do not paper over

When you spot a contradiction, surface it explicitly: 'turn 12 you said X; turn 41 you said not-X; reconcile with sources.' Asking the model to reconcile with sources forces it to ground in something durable rather than pick the most recent answer by default.

Self-disagreement maps drift

If the same question gets different answers at turn 12, 41, and 73, you have a drift map: facts that depend on recent context, not on stable rules or sources. Move those facts to durable storage if they should be invariant; accept the variation if they are genuinely turn-specific.

Code

Reconcile prompt·text
Earlier in this session:
  Turn 12: you said X.
  Turn 41: you said not-X.
Reconcile by citing sources from the spine, the body, or the current source files.
If both answers were unsourced model priors, say so and pick neither until verified.
Disagreement log·yaml
self_disagreement:
  topic: "DB connection pooling default"
  answers:
    - turn: 12, claim: "pool size 10", source: "unverified"
    - turn: 41, claim: "pool size 20", source: "backend/db.py:18"
  resolution: "trust turn 41 (sourced); add explicit rule to spine"

External links

Exercise

Pick a fact you have seen the model handle inconsistently. Build a reconcile prompt for it. Run it. Decide where the fact should live to stop the inconsistency.
Hint
Inconsistencies usually mean the fact is missing from durable storage.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.