The Outbox: A Capture Is Never Lost

"Look in the place nothing points at."

The Local Write Is the Capture

A note spoken on a trail, a photo in a tunnel, a set logged in a basement gym: family phone apps capture where there is no network, so a capture cannot depend on the engine. The outbox makes the local write the capture and the send a separate act that may fail. Each entry carries an id the device minted, and the engine answers a replay of that id by returning the record it already has instead of appending a second one. That one rule makes retrying safe everywhere else.

An entry moves through four states that matter: QUEUED, SYNCING (the one in flight), SYNCED and FAILED_NEEDS_ATTENTION. An app killed mid-send finds an entry stuck in SYNCING at launch and simply puts it back in line; the engine's dedupe makes the second send harmless.

How a Drain Decides

A drain walks the queue in order. A permanent failure (the engine refuses this payload) parks that entry where the user can see it and continues, because it says nothing about the entries behind it. A transient failure puts the entry back and stops, because the rest would fail the same way. The line between them was wrong in both Swift apps the shared version came from: each called every 4xx permanent. But 401 and 403 mean a wrong session, not a wrong payload, and 408, 425 and 429 mean "not now". An app that knows its own engine can refine further, like the travel journal that treats its engine's 405 as "restart the engine and retry".

Idempotent at Both Ends

The training app's watch re-offers its captures to the phone whenever the link comes back, by design, and the phone's store appended every offer. One acknowledgement settled one copy, the other sat at the head of the queue for eight builds, and a completion step that returned quietly for an id the store no longer held was the other half of a loop that sent 774 POSTs. The fix has three parts: enqueue is idempotent on the id, duplicates already on disk are healed at load (a settled copy wins), and marking an unknown id as synced throws.

Two Lists, and a Count That Must Not Drop

The travel journal's photos stayed on the phone forever after their entries had synced. One list did two jobs: the files still to upload, and the files to delete when the entry retires. The drain correctly empties the first as uploads land, so by acknowledgement time the cleanup loop iterated nothing. Ownership is its own field that is never consumed, and the test must drain the pending list to empty before retiring, or it passes against the bug.

The last trap is the quietest. A store that lists its entries with try? decode and continue turns one added field into an empty queue: every manifest written by the previous build stops decoding at once, and the app says it has nothing to send. Skipping is right; dropping the count is the bug. Return the unreadable entries beside the readable ones, show "N captures this app cannot read, still on this device", and decode old entries with defaults chosen so nothing is lost: QUEUED for the state, the epoch for a missing creation date so a rescued capture drains first.

Code

An outbox: idempotent enqueue, launch recovery, and a drain that knows when to stop·swift

import Foundation

enum OutboxState: String, Codable, Sendable {
    case queued = "QUEUED"
    case syncing = "SYNCING"
    case synced = "SYNCED"
    case needsAttention = "FAILED_NEEDS_ATTENTION"
}

struct OutboxEntry: Codable, Sendable, Equatable {
    let id: String               // minted on the device; the engine answers a replay of it idempotently
    var state: OutboxState
    var pendingMedia: [String]   // still to upload, consumed as each upload lands
    let ownedMedia: [String]     // what to delete on every exit path, never consumed
    var lastError: String?
}

enum SendFailure: Error, Equatable {
    case transient(String)
    case permanent(String)
}

/// A wrong session or a busy engine is a wait. Another 4xx refuses THIS payload.
func classify(status: Int) -> SendFailure? {
    switch status {
    case 200..<300: nil
    case 401, 403, 408, 425, 429: .transient("HTTP \(status)")
    case 400..<500: .permanent("HTTP \(status)")
    default: .transient("HTTP \(status)")
    }
}

struct Outbox {
    private(set) var entries: [OutboxEntry] = []

    /// Idempotent on id: a payload offered twice stays one entry.
    mutating func enqueue(_ entry: OutboxEntry) {
        guard !entries.contains(where: { $0.id == entry.id }) else { return }
        entries.append(entry)
    }

    /// At launch. A send killed mid-flight goes back in line; the engine dedupes the replay.
    mutating func recoverInterruptedSends() {
        for index in entries.indices where entries[index].state == .syncing {
            entries[index].state = .queued
        }
    }

    /// One pass in order. Permanent: park that entry and continue. Transient: requeue and stop.
    mutating func drain(send: @Sendable (OutboxEntry) async -> Int) async -> (synced: Int, haltedBecause: String?) {
        var synced = 0
        for index in entries.indices where entries[index].state == .queued {
            entries[index].state = .syncing
            switch classify(status: await send(entries[index])) {
            case nil:
                entries[index].state = .synced
                synced += 1
            case .permanent(let reason)?:
                entries[index].state = .needsAttention
                entries[index].lastError = reason
            case .transient(let reason)?:
                entries[index].state = .queued
                return (synced, reason)
            }
        }
        return (synced, nil)
    }
}

A check run: one id offered twice, one send interrupted, then 200, 422, 503, 200·text

entries after double offer: 4
states after recovery: ["QUEUED", "QUEUED", "QUEUED", "QUEUED"]
outcome: (synced: 1, haltedBecause: Optional("HTTP 503"))
states after drain: ["01A=SYNCED", "01B=FAILED_NEEDS_ATTENTION", "01C=QUEUED", "01D=QUEUED"]

Exercise

Put the outbox in SparkKit and reproduce the check run in a self-test. Then add the two missing behaviors: markSynced(id:) that throws for an id the store does not hold, and a retire(id:) that deletes ownedMedia files from a temporary directory. Write the retire test the honest way: upload every pending file first so pendingMedia is empty, retire, and assert the files are gone. Finally, make the store load a JSON document where one entry is missing its state key, and report it rather than dropping it.

Hint

For the missing-key case, give OutboxEntry a hand-written init(from:) that uses decodeIfPresent with a default of .queued, and keep a separate list of entries that failed for a reason you cannot default, such as a missing id.