C.W.K.
Stream
Lesson 04 of 04 · published

Ground Truth and the Dev Mirror

~11 min · ground-truth, dev-mirror, model-storage, one-way-sync

Level 0Tool Renter
0 XP0/33 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"'It's not on my machine' is not the same as 'it doesn't exist.' One of those is a fact; the other is a guess about your dev mirror."

Two Machines, One Authoritative

The engine runs in two places: a server that is the authoritative model store and serving host, and a development machine that mirrors the models for local work. They're the same hardware, but they play different roles. The server is ground truth — the place models actually live. The dev machine is a mirror — a copy that may or may not be complete at any given moment.

Sync Flows One Way

The rule that makes this safe: model files flow in exactly one direction. A new model lands on the server first, then propagates to the dev mirror. Never the reverse. This one-way flow means the server is always the most complete, most current store, and the dev machine is always somewhere between 'empty' and 'fully caught up.' Knowing the direction tells you which machine to trust when they disagree.

Name your ground truth, and make sync flow toward the mirror, never from it. When the same data lives in two places, designate one as authoritative and enforce one-way propagation. The mirror can lag, be incomplete, or be wiped and rebuilt — none of that threatens the ground truth, because nothing flows back from the mirror.

The Bug This Prevents

Here's the mistake the rule exists to stop. You're on the dev machine, you look for a model, it's not there, and you conclude 'this model doesn't exist' — then you write code, file a bug, or make a decision based on that false conclusion. But the dev machine is just mid-sync; the model exists on the server, ground truth, and simply hasn't propagated yet. The dev mirror showing fewer models is normal, not evidence of absence.

An incomplete mirror looks identical to a missing item. 'Not present in my local copy' and 'does not exist anywhere' produce the exact same observation locally — emptiness. You cannot tell them apart from the mirror alone. The only way to know is to check ground truth. Treat every 'it's not here' on a mirror as 'check the source' until proven otherwise.

Why Not Just Sync Everything Always?

You might want the mirror to be a perfect, instant copy — but model files are enormous, and full propagation takes real time. During that time the mirror is legitimately incomplete. Rather than pretend the mirror is always perfect (and get burned by the gap), the discipline accepts that the mirror lags and builds the habit of checking ground truth before concluding absence. Accepting the lag and accounting for it beats pretending it isn't there.

Design for the lag you can't eliminate. Some gaps — sync time for huge files, eventual consistency, propagation delay — can't be engineered away. The robust move is to make the lag visible and build habits around it (check the source), not to pretend the mirror is instantaneous and be surprised by the gap. Honest about the lag beats optimistic and wrong.

Pippa's Confession

I got burned by exactly this. I checked the dev machine, didn't see a model, and confidently reported it missing — when it was sitting on the server the whole time, just mid-sync to my mirror. Dad's rule is now reflex: 'not on the dev mirror' triggers 'check the server,' every time, before I conclude anything. The humbling part was how certain I'd been. Emptiness on a mirror is the most confident-looking lie there is.

Code

One-way flow names the trustworthy machine·text
  SERVER (ground truth)              DEV MACHINE (mirror)
  authoritative model store          local copy for development
  always most complete       --->    may be incomplete / mid-sync
          |                          
          |   one-way propagation    
          +------------------------> (never flows back)

  Decision rule when a model 'isn't there':
    on the dev mirror?  -> NOT proof of absence. Check the server.
    on the server?      -> THIS is ground truth. Now you know.

  'not on my machine'  !=  'does not exist'

External links

Exercise

Identify a place in your work where you have an authoritative source and one or more mirrors (a prod DB and local copies, a canonical config and cached versions, a main branch and working copies). When something 'isn't there' in a mirror, does your process check the source before concluding absence? If not, that's a latent 'confidently wrong' bug.
Hint
The test is: has anyone on the team ever concluded something was missing/broken/gone based on a mirror, only to find it fine at the source? If yes, you've already paid for this lesson once.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.