~14 min · migration, source-of-truth, idempotency, audit
Level 0Sketch
0 XP0/43 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete
The handover problem
Before the studio, Pippa's avatar folders were the originals. There was no other copy to be canonical. So Avatars could not start empty and claim authority; it had to take over the folders that already existed, every byte intact, without inventing a history for pictures it never made, and then close the door behind it so the old direction could never reopen. The architecture states the result: Avatar authority is one-way: Avatars → Pippa. This lesson is how that line became true, once, and how it stays true.
The import that runs once
Studio.import_existing starts by asking whether an ownership record exists. If it does and its state is canonical, the import does no importing at all and returns an audit of the copies. If it exists and is still adopting, it finishes that adoption. Only with no ownership record does it read Pippa's legacy catalog: every soul's collections, each frame with its path, SHA-256 and the folder's revision. Pippa's docstring for that catalog calls it a legacy bootstrap inventory only, never a source for studio references.
For each legacy collection, the import skips any set Avatars already has, with the comment importing never resets Dad's current selections. For the rest it reads every frame, checks that the path stays inside Pippa's avatar folders, and compares the bytes with the catalog's hash: An existing avatar changed during import. Retry the import. Each frame becomes an original with origin imported, bound to its soul. The new set gets the same map as its selection and as its published frames, and the catalog's revision as its live revision, because that is exactly what Pippa is serving.
Then it commits an ownership intent under a fixed request id, legacy-avatar-authority-v1, in state adopting, listing every set that has published frames and no publication record. A fixed id makes the intent itself idempotent: a second run finds the same record.
Adopt mode installs nothing
finish_adoption publishes each listed set with request id adopt-<set id> in adopt mode, resuming any that is already pending, and then marks the ownership canonical. In adopt mode Avatars publishes the set's published map against the revision it recorded, so a copy that changed since the import is refused like any stale preview.
On Pippa's side, adopt mode never stages or swaps. It checks that every file in the live folder hashes to the frame it claims (Legacy copies do not match the Avatars originals), backs the folder up, writes a prepared receipt marked legacy-adoption, and on finishing checks the revision once more and writes only avatar-lineage.json into the existing folder. Pippa's test asserts that the file names, the modification time and the bytes are all unchanged afterwards, and that no PNG appeared. The publication records say legacy-adoption too: the architecture insists the adoption does not invent a historical generation or an Ember receipt.
The README records the outcome: Legacy adoption retained 301 images in 32 collections, with baseline publication receipts and byte-identical Pippa copies.
After canonical, drift is a report
The import's entry point in the WebUI is the sidebar button Verify published copies. Once the ownership is canonical, that button only audits. copy_status recomputes each published set's revision from Avatars' own recorded hashes, using the formula Pippa uses, and asks Pippa for the live revision. Equal means the copy is faithful. Different is reported as N published collections differ from their Avatars originals. Review and republish the affected collections. It never reads the changed files and never imports them. The docstring says it plainly: compare installed copies without ever importing their changes.
The adoption test pins every part of that. A first import fails after Pippa applied; the ownership is still adopting. A second import completes it as canonical, with the identical payload sent to Pippa, and the legacy catalog read exactly once. Then the test makes every copy look missing. The audit reports all of them as not matching, the catalog is still read only once, the set's published map is unchanged, and the soul's reference profile still resolves the original. A broken copy changes a report, and nothing else.
The one door left open, on purpose
Registered people joined the avatar boundary later the same night, after the main adoption had closed, so their existing folders were never part of it. Reopening the import would have broken the rule for everyone. Instead there is backend/adoption.py, an explicit, resumable adoption for one registered subject omitted at cutover. scripts/adopt-subject.py <subject> prints a preview, and only --apply writes. It refuses a subject that already has any Avatars collection (legacy copies cannot replace or extend them), freezes the source hashes in its own intent, rechecks every frozen original before any write on every run, and ends in the same finish_adoption. It is a migration entry point, not a button anyone presses in daily work.
Principle: authority moves once, through a door that records itself and then locks. After that, a difference between owner and copy is something to report, never something to merge.
Pippa Quest asks this from the other side, in its soul track: a frame in Pippa's folder that differs from what Avatars published is drift, and the repair is a publication from Avatars.
Code
cwkAvatars, backend/studio.py: import_existing, verify every legacy byte and commit the ownership intent (verbatim, lines 155-185)·python
async def import_existing(self):
"""One-time transfer of authority; subsequent calls only audit copies."""
ownership = await self.io("list", "ownership")
if ownership:
if ownership[0]["state"] == "canonical":
return {"imported": [], "ownership": ownership[0], "copies": await self.copy_status()}
return await self.finish_adoption(ownership[0])
registry = await self.upstream.legacy_catalog()
imported = []
async with self.intent_lock:
current_sets = await self.io("list", "set")
for source in registry["sets"]:
current = next((s for s in current_sets if (s["soul_pk"], s["channel"], s["name"]) == (source["soul_pk"], source["channel"], source["name"])), None)
if current:
continue # importing never resets Dad's current selections
frames = {}
for frame, item in source["frames"].items():
path = Path(item["path"]).resolve()
path.relative_to((Path(__file__).resolve().parents[2] / "cwkPippa/avatars").resolve())
data = await asyncio.to_thread(path.read_bytes)
if hashlib.sha256(data).hexdigest() != item["sha256"]:
raise Conflict("An existing avatar changed during import. Retry the import.")
asset = await self.io("save_asset", data, label=f"{source['name']} · {frame}", subjects=[source["soul_pk"]], origin="imported", source_path=str(path))
frames[frame] = asset["id"]
record = await self.io("add", "set", soul_pk=source["soul_pk"], channel=source["channel"], name=source["name"], frames=list(frames), published=frames, selection=frames, live_revision=source["revision"], archived=False)
imported.append(record)
collections = await self.io("list", "set")
legacy = [s["id"] for s in collections if s.get("published") and not await self.io("list", "publication", s["id"])]
ownership = await self.io("create_intent", "legacy-avatar-authority-v1", {"schema": 1}, "ownership", {"state": "adopting", "owner": "cwkAvatars", "legacy_sets": legacy, "started_at": now()})
result = await self.finish_adoption(ownership)
return {**result, "imported": imported}
cwkAvatars, backend/studio.py: finish_adoption and copy_status, publish each set once in adopt mode, then only compare (verbatim, lines 187-216)·python
async def finish_adoption(self, ownership):
for id in ownership["legacy_sets"]:
request_id = f"adopt-{id}"
records = await self.io("list", "publication", id)
existing = next((p for p in records if p.get("payload", {}).get("request_id") == request_id), None)
if existing:
if existing["state"] != "applied":
receipt = await self.upstream.publish(existing["payload"])
await self.io("complete_publication", existing["id"], receipt)
continue
collection = await self.io("get", id, "set")
await self.publish(id, request_id, collection["version"], collection["live_revision"], mode="adopt")
ownership = await self.io("update", ownership["id"], state="canonical", completed_at=now())
return {"ownership": ownership, "imported": []}
async def copy_status(self):
"""Compare installed copies without ever importing their changes."""
collections = [c for c in await self.io("list", "set") if c.get("published")]
ids = list(dict.fromkeys(id for c in collections for id in c["published"].values()))
lineage = await self.io("asset_lineages", ids) if ids else {}
# One registry round trip covers every published subject; the policy
# verdict is still the registry's, taken now.
await self.subjects(sorted({s for a in lineage.values() for s in a["subjects"]}))
answer = []
for collection in collections:
hashes = {frame: lineage[id]["sha256"] for frame, id in sorted(collection["published"].items())}
revision = hashlib.sha256(json.dumps(hashes, sort_keys=True).encode()).hexdigest()
actual = await self.upstream.revision(collection["soul_pk"], collection["channel"], collection["name"])
answer.append({"set_id": collection["id"], "soul_pk": collection["soul_pk"], "channel": collection["channel"], "name": collection["name"], "expected_revision": revision, "actual_revision": actual, "matches": actual == revision})
return answer
cwkPippa, backend/services/avatar_studio.py: publish in adopt mode, the copies must already match and only a receipt is prepared (verbatim, lines 237-252)·python
if body.get("mode") == "adopt":
actual = {frame: digest(path) for frame, path in frame_files(target).items()}
if actual != {frame: item["sha256"] for frame, item in files.items()}:
raise ValueError("Legacy copies do not match the Avatars originals")
backup = REPO / "avatar-backups" / "studio" / request_id
backup.mkdir(parents=True, exist_ok=True)
if target.exists():
if not (backup / "previous").exists():
shutil.copytree(target, backup / "previous")
if revision(frame_files(backup / "previous")) != previous:
raise ValueError("Backup verification failed")
if body.get("mode") == "adopt":
manifest["revision"] = previous
receipt = {"request_id": request_id, "fingerprint": fingerprint, "status": "prepared", "mode": "adopt", "provenance": "legacy-adoption", "soul_pk": body["soul_pk"], "channel": body["channel"], "name": body["name"], "previous_revision": previous, "revision": previous, "backup": str(backup), "target": str(target), "manifest": manifest}
atomic_write_text(receipt_path, json.dumps(receipt, indent=2))
return finish_publication(receipt, receipt_path)
Plan a one-time handover of authority in a system you know, where a legacy store holds data a new service must own. Write the import so it verifies every item against a hash taken at inventory time, records a single ownership intent under a fixed id with an adopting state, and finishes by marking it canonical. After that, make the same entry point an audit that reports differences without reading them in. Write a test that fails the import halfway, reruns it, and checks the inventory was read exactly once.
Hint
The fixed id is what makes the handover happen once: a second run finds the intent and resumes it instead of starting over. Keep a separate, explicit, preview-first path for anything forgotten at cutover, and make it refuse targets the new service already owns.
Progress
Progress is local-only — sign in to sync across devices.