C.W.K.
Stream
Lesson 03 of 05 · published

Stage, Verify, Publish

~12 min · staging, publish, atomicity

Level 0Lost in Finder
0 XP0/36 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"Never let a half-written file wear the real file's name. Give it a costume until it's proven, then let it change into the real thing at once."

The Problem with Writing Straight to the Destination

Imagine copying report.pdf directly to its final path on the destination volume. Halfway through, the volume drops. Now there's a file named report.pdf at exactly the right location — but it's truncated garbage. Nothing on disk distinguishes it from a finished copy. The next person to open it, or the next tool to sync it, trusts a corpse. Writing straight to the destination name is how partial copies masquerade as real files.

Staging: a Costume Until Proven

Waygate copies into an operation-named staging item on the destination volume — a temporary name tied to the operation id, clearly not the final file. During the copy it preserves what the plan requires: supported metadata, resource forks, ACLs, extended attributes, timestamps. Then it verifies the staged result against the plan's expectations. Until that verification passes, the real destination name does not exist — there is nothing there to mistake for success.

Publish: the Costume Comes Off at Once

Only after verification does Waygate publish: it makes the staged file appear at the real destination name, using a same-volume rename where possible. Because the staging item and the destination are on the same volume, that rename is effectively atomic — the file either has its real name or it doesn't, never halfway. The moment the real name appears, the file behind it is already complete and verified. Publish is the instant the copy becomes true.

Copy into operation-named staging, verify, then publish with a same-volume rename. A half-written file never occupies the real destination name. Staging keeps the incomplete copy under an operation-owned name; verification gates the publish; the atomic rename makes the real name appear only when the bytes behind it are already correct.
Staging on the destination volume is the trick. The staging item must live on the SAME volume as the final destination, because that's what makes the publish a cheap atomic rename instead of another cross-volume copy. Staging on the source volume, then copying again, would just reintroduce the exact non-atomic risk staging exists to remove.

Code

Stage under an op-owned name, verify, then atomic publish·swift
func copyThenPublish(_ src: URL, to dest: URL, _ plan: OperationPlan) async throws -> URL {
    // Staging item lives on the DESTINATION volume, named for the operation:
    let staging = dest.deletingLastPathComponent()
        .appendingPathComponent(".waygate-stage-\(plan.id)")

    try await copyPreservingMetadata(src, to: staging)   // forks, ACLs, xattrs, dates
    try await verify(staging, matches: plan.expectedMetadata)
    try await journal.mark(plan.id, .verified)

    // Same-volume rename -> effectively atomic. The real name appears
    // only now, and the file behind it is already complete + verified.
    try FileManager.default.moveItem(at: staging, to: dest)
    return dest
}
// A crash before the rename leaves only `.waygate-stage-<id>` -- an obviously
// incomplete artifact the journal owns, never a fake `report.pdf`.
The costume, then the reveal·text
  during copy:   .waygate-stage-op_7a3f    <- obviously not the real file
  crash here?    only the staging item exists; report.pdf never appeared

  verify passes: bytes confirmed against the plan
  publish:       .waygate-stage-op_7a3f  --(atomic rename)-->  report.pdf

  after publish: report.pdf exists AND is already complete + verified.
                 There is no instant where report.pdf is half-written.

External links

Exercise

Think about a download that got interrupted and left a file that looked complete but was corrupt — a truncated video, a broken zip. Now redesign that download using stage-verify-publish: what name does it write to during the download, what does it check, and when does the real filename appear? Why can this version never leave a corrupt file at the real name?
Hint
Write to something like file.part during the download, verify the size or checksum when it finishes, and only then rename file.part to file. Because the real name appears exclusively via a post-verification rename, an interrupted download leaves only file.part — an obviously-incomplete artifact — and never a corrupt file wearing the real name. That's exactly Waygate's staging discipline.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.