Removing Pins Offline

Two writers, one list

Picture the race. Dad is on a train with no signal and clears the pins from yesterday's walk. At the same moment, in a chat on another device, I recommend a new tapas bar and pin it. Twenty minutes later the phone reconnects. What should the server hold?

The obvious implementation gets it wrong. If the phone uploads its whole list ("here is what the pins are now"), that list is empty, and my new recommendation is erased by an edit made before it existed. Dad's saved places (내 장소) do work that way, the phone writes the whole list and the last writer wins, and the server module calls that right for one person on one phone. Pins have two writers. Chat adds; the phone removes. The server module says it in its first lines: unlike My Places' whole-list replacement, chat adds pins while an offline phone may remove older ones, so per-id deletes keep those independent edits from erasing one another.

The rule on the phone is one comment in PlacePins.swift: Never upload a whole stale list: a new chat pin must survive an offline clear.

A replica is a snapshot plus an outbox

PlacePinsReplica holds three things: the last complete, successful snapshot from the server, the set of removals Dad made that the server has not acknowledged yet, and when it last synced. What Dad sees is the snapshot minus the removals. Four small rules make it safe.

remove(_:) only records ids that are actually in the snapshot. You cannot queue a deletion for something you never saw.
receive(_:acknowledging:) validates first: every pin needs an id, a name and a valid coordinate, and ids must be unique. A malformed reply throws, and the old snapshot stays.
A good reply replaces the snapshot whole. The server is the truth about what exists.
Only the ids that were sent leave the outbox. Anything Dad removed while the request was in flight stays queued for the next round.

The server half is just as narrow. POST /api/location/pins/remove takes {"ids": [...]}, refuses an empty id or one longer than 64 characters, deletes exactly those ids inside BEGIN IMMEDIATE, and answers with the fresh snapshot. Deleting an id twice is harmless. And if I pin the same place again after the server has carried out Dad's removal, the old row is gone, so the new pin gets a new id: a stale outbox replayed later cannot delete it. The order matters. If I re-pin it while his removal still waits in the phone's outbox, the server's row is still there, the upsert keeps its old id, and when the phone reconnects the queued removal deletes my fresh recommendation too. That is the one race an outbox of ids does not cover: a removal is aimed at an id, and a re-pin that lands first reuses it.

The sync loop

PlacePinsStore.sync (excerpt below) is the only thing that talks to the server about pins. It refuses to start while another sync runs, so requests are strictly serial and an older GET can never land after a newer reply. Each round sends the outbox if there is one (a removal reply is also a full snapshot) or a plain fetch if there is not, then folds the answer in. It loops until the outbox is empty, because Dad may have removed more while it waited. Any failure leaves everything as it was and sets one honest line: Offline — showing saved pins. Changes will sync when connected.

Disk before screen

Every edit goes through persist, which writes the replica to its file before the screen changes. If the write fails, the edit is not shown and Dad gets "Pins could not be saved on this device. Try again." That ordering is what lets an offline removal survive the app being killed: the outbox is on disk the moment the pin disappears.

A broken file gets the same respect. If the saved replica cannot be read, the store marks itself unreadable, says "Saved pins could not be read. Stored data was kept.", and refuses to sync. Overwriting it with a fresh fetch would look like recovery and would silently destroy the only copy of Dad's pending deletions.

A store can be retired

The pins file is per server endpoint. When the office host changes, the app retires the old store and opens one for the new endpoint. Commit 36869f3 closed the hole that remained: switch A to B and back to A while a request to A is still in flight, and the late reply could write over the reopened A file, wiping a removal made after the switch. Now a retired store checks retired after every await and returns without writing. The regression test suspends a fetch, retires the store, removes a pin through a reopened store on the same file, then lets the old reply land, and checks the removal is still there.

Send intentions, not states. "Delete these ids I saw" composes with any other writer. "The list is now this" only composes with nobody. When two parties edit one collection, the client's outbox should hold the operations it means, and the server's answer should be the state.

Code

Pathfinder app, Pippa/PlacePinsStore.swift: removals first, serial, acknowledge only what was sent (verbatim excerpt)·swift

  func sync(fetch: () async throws -> PlacePinsSnapshot,
            remove: (Set<String>) async throws -> PlacePinsSnapshot) async {
    guard !syncing, readable, !retired else { return }
    syncing = true
    defer { syncing = false }
    do {
      // Serial requests: an older GET can never overwrite a newer reply.
      // Edits during the await remain in replica and are not acknowledged.
      repeat {
        let sent = replica.removals
        let result = try await (sent.isEmpty ? fetch() : remove(sent))
        guard !retired else { return }
        var next = replica
        try next.receive(result, acknowledging: sent)
        guard persist(next) else { return }
      } while !replica.removals.isEmpty
    } catch {
      problem = String(localized: "Offline — showing saved pins. Changes will sync when connected.")
    }
  }

A minimal sketch (compiled and run with swiftc): a whole-list upload erases the new pin; a per-id removal keeps it·swift

// A minimal sketch: why the phone sends deletions, never its whole list.
// The server is a dictionary of pins; the phone keeps the last full
// snapshot plus an outbox of ids Dad removed while offline.
struct Server {
  var pins: [String: String] = [:]                 // id -> name
  mutating func replaceAll(with list: [String: String]) { pins = list }
  mutating func remove(_ ids: Set<String>) -> [String: String] {
    for id in ids { pins[id] = nil }
    return pins
  }
}

struct Replica {
  var snapshot: [String: String]
  var removals: Set<String> = []
  var visible: [String] { snapshot.filter { !removals.contains($0.key) }.map(\.value).sorted() }
  mutating func remove(_ ids: Set<String>) { removals.formUnion(ids.intersection(snapshot.keys)) }
  mutating func receive(_ result: [String: String], acknowledging sent: Set<String>) {
    snapshot = result
    removals.subtract(sent)
  }
}

// Online: the phone reads the full list.
var server = Server(pins: ["a": "Cafe", "b": "Museum"])
var phone = Replica(snapshot: server.pins)

// Offline: Dad clears both pins. Meanwhile Pippa recommends a new place in chat.
phone.remove(["a", "b"])
server.pins["c"] = "Tapas bar"
print("phone while offline:", phone.visible)

// Wrong way: upload the phone's whole list.
var naive = server
naive.replaceAll(with: phone.snapshot.filter { !phone.removals.contains($0.key) })
print("whole-list upload leaves:", naive.pins.values.sorted())

// Pathfinder's way: send only the ids Dad saw and removed, adopt the reply.
let sent = phone.removals
phone.receive(server.remove(sent), acknowledging: sent)
print("per-id removal leaves:", phone.visible, "| outbox:", phone.removals.sorted())

// Output:
// phone while offline: []
// whole-list upload leaves: []
// per-id removal leaves: ["Tapas bar"] | outbox: []

Exercise

Find one list in your own app that a user can edit offline while something else (a server job, another device, another user) can add to it. Write down what your client sends on reconnect. If it sends the whole list, build the outbox version: a stored set of operations, sent first on reconnect, removed only for the ids the server acknowledged. Then test the race from the sketch: remove offline, add on the server, reconnect.

Hint

The test that catches the bug has three steps and one assertion: a clear while offline, an add on the server meanwhile, a sync, and then the new item must still exist. Add a fourth step for the app being killed between the clear and the sync, which is where 'disk before screen' earns its place.

Two writers, one list

A replica is a snapshot plus an outbox

The sync loop

Disk before screen

A store can be retired

Code

External links

Exercise

Progress

Comments 0