Lesson 03 of 04 · published

The Drive Log Is Evidence

~14 min · logging, field-testing, evidence, testflight

Level 0Lost

0 XP0/36 lessons0/12 achievements

0/100 XP to next level100 XP to go0% complete

A live experiment needs a lab notebook

On the first day Dad set the terms: treat the app as a live experiment, and record what works and what does not. The house sign turns that into a rule: the real drive is the test that counts. Simulator replays prove the wiring; the drive log of an actual drive is the evidence.

That split matters. A simulator can replay a route along Apple's own geometry and walk the whole guidance loop, and Pathfinder does exactly that. What it cannot give you is a tunnel that eats the GPS, a parking garage start, a phone call in the middle of a turn, or a watch that quietly stops listening. The road has those. The drive log is how the road reports back.

What one drive writes

DriveLog lives in PathfinderKit, under test like the rest of the brain. It writes one JSON object per line, appended the moment each thing happens. There is no in-memory buffer waiting for a clean shutdown that may never come: each line goes to the operating system as soon as it is encoded, so an app that crashes or is killed mid-drive still leaves every line up to that moment.

The source's own comment promises more, "a crash or a dead battery", and the battery half is not proven. append never calls synchronize(); only close() does. A line the system has accepted but not yet put on storage can still be lost if the power goes all at once, and no test on a Mac can pull the power. Until a drive shows otherwise, that half is a hope, not a property.

Event kinds: start, route, fix, announce, camera, off_route, reroute, back_on_route, arrived, end, spoke, voice_fallback, handoff, note, error.
Fixes are thinned to at most one a second; every other event is always kept.
One file per drive, drive-<start time in Seoul>.jsonl, in a drives folder on the phone. It is never uploaded. Settings lists every drive as a share link, so Dad can export one when a drive needs reading.
What it keeps is Pathfinder's own. The fixes are its measurements; route text appears only as what was spoken to Dad, like a conversation record. A trip Google guides writes google:guided by the provider and nothing of the road, because Google's words stay on Google's map.
What is not a drive leaves no file. A trip that never started is deleted on the spot, and a simulation writes nothing at all.

The road keeps finding things

The habit is simple: build, take it out, bring back what happened, fix, build again. A few of the things the road found:

The first TestFlight test: on-device MapKit answered Hangul queries with "not found" (MKErrorDomain 4), and "hufs" searched near Ilsan answered Hays, Texas. Search got a centre and Pippa's Apple Maps door (lesson 4).
The first drives: Dad asked for his own places, a flat heading-up map with him low on the screen, and a walking mode. One commit built all three.
Build 7 went out on the road and came back with Dad's verdict, "테스트했어. 잘 돼." ("Tested it. It works."). Todo #290 closed on his word, not on a green suite.
Walking to Madrid's Atocha station, Dad saw the next turn, the arrival time and the distance, but never where he was going (#367). The guidance bar now names the destination on every stack.
In Seville the watch followed nothing (#391), and no record said why.

When the log is silent, that is a finding too

The Seville case is the one worth sitting with. The watch was fixed. But the more durable part of the fix is in the log: the phone now writes a note when the watch stops hearing the guidance and another when it hears it again. A drive that starts with the watch connected says nothing, so the log does not fill with noise.

The same instinct shows up wherever a failure could otherwise vanish. When Google's turn feed never arrives, Pathfinder hands the turns to Google's own voice and writes google: no turn feed; its own voice guides. When a line had to fall back to the system voice, voice_fallback says so. A record that only covers the happy path is a diary, not evidence.

Fix the bug, then fix the record that missed it. If a failure on the road left no trace, the change that repairs it also adds the event that would have shown it. The next silence then arrives with a timestamp.

Builds as the heartbeat

The loop only works if new builds reach Dad's phone quickly. Pathfinder's pipeline commits run from TestFlight build 2 to build 30 in its first twelve days. Every one went through TestFlight: in this family an iOS app reaches Dad's devices that way, and a cable install is a build check, not delivery.

Even a pipeline commit is not the same as "Dad has it". Upload, Apple's processing, assignment to the internal testing group, and installation on his phone are four separate facts. The first three can be checked from a Mac; the fourth only his phone can witness. The release gate for 1.0 asks for it explicitly, alongside a real drive from search to arrival with a reroute, a camera warning, and Pippa answering from the navigation report.

Code

cwkPathfinder, PathfinderKit/Sources/PathfinderKit/Drive/DriveLog.swift: append (verbatim)·swift

  public func append(_ event: DriveEvent) {
    if event.kind == .fix, let at = ISO8601DateFormatter.fractional.date(from: event.at) {
      if let lastFixAt, at.timeIntervalSince(lastFixAt) < fixInterval { return }
      lastFixAt = at
    }
    guard var data = try? encoder.encode(event) else { return }
    data.append(0x0A)
    try? handle?.write(contentsOf: data)
  }

A minimal sketch of the same idea: append-only JSONL, fixes thinned, read back with no close (compiled with swiftc and run; output at the end)·swift

import Foundation

// One JSON line per event, written the moment it happens. Fixes are thinned to
// one a second; every other event is always kept. Each line goes to the OS as
// soon as it is encoded, with no buffer in the app, so a crash keeps every line
// written so far. Nothing calls synchronize(), so a sudden power cut is a
// different question that this sketch does not answer.

struct Event: Codable {
  var at: Date
  var kind: String
  var text: String?
}

final class DriveLog {
  let url: URL
  private let handle: FileHandle
  private let encoder = JSONEncoder()
  private var lastFixAt: Date?
  private let fixInterval: TimeInterval = 1

  init(url: URL) throws {
    FileManager.default.createFile(atPath: url.path, contents: nil)
    handle = try FileHandle(forWritingTo: url)
    encoder.dateEncodingStrategy = .secondsSince1970
    encoder.outputFormatting = [.sortedKeys]
    self.url = url
  }

  func append(_ event: Event) {
    if event.kind == "fix" {
      if let lastFixAt, event.at.timeIntervalSince(lastFixAt) < fixInterval { return }
      lastFixAt = event.at
    }
    guard var data = try? encoder.encode(event) else { return }
    data.append(0x0A)
    try? handle.write(contentsOf: data)
  }
}

let start = Date(timeIntervalSince1970: 0)
func at(_ s: Double) -> Date { start.addingTimeInterval(s) }

let url = FileManager.default.temporaryDirectory.appendingPathComponent("drive-sketch.jsonl")
let log = try DriveLog(url: url)
log.append(Event(at: at(0), kind: "start", text: "to the station"))
for s in stride(from: 0.0, through: 6.0, by: 0.4) { log.append(Event(at: at(s), kind: "fix")) }
log.append(Event(at: at(2.0), kind: "spoke", text: "300미터 앞에서 우회전해"))
log.append(Event(at: at(4.4), kind: "off_route", text: "lateral 62 m"))
log.append(Event(at: at(5.1), kind: "reroute"))
log.append(Event(at: at(9.8), kind: "back_on_route"))
// No close() and no synchronize(): read back what the file already holds.
// This shows that nothing waits for a clean close. It is not a crash or a
// power cut; the exercise below kills a real process.

let decoder = JSONDecoder()
decoder.dateDecodingStrategy = .secondsSince1970
let lines = try String(contentsOf: url, encoding: .utf8).split(separator: "\n")
let events = lines.compactMap { try? decoder.decode(Event.self, from: Data($0.utf8)) }
let fixes = events.filter { $0.kind == "fix" }.count
print("lines on disk: \(events.count) (fixes kept: \(fixes) of 16 offered)")
if let off = events.first(where: { $0.kind == "off_route" }),
  let back = events.first(where: { $0.kind == "back_on_route" })
{
  print("off the route for", String(format: "%.1f s", back.at.timeIntervalSince(off.at)))
}

// Output:
// lines on disk: 11 (fixes kept: 6 of 16 offered)
// off the route for 5.4 s

cwkPathfinder, clients/apple/PathfinderMobile/Sources/App/AppModel.swift: the watch's silences go into the drive log, #391 (verbatim)·swift

  /// A drive's log says when the watch stops hearing the guidance and when
  /// it hears it again (todo #391: in Seville the watch followed nothing and
  /// no record said why). A drive that begins with the watch heard says
  /// nothing.
  private func noteWristTrouble(_ trouble: String?) {
    guard !isSimulating, let log = driveLog else { return }
    let id = ObjectIdentifier(log)
    if let note = wristNote, note.log == id, note.trouble == trouble { return }
    let first = wristNote?.log != id
    wristNote = (id, trouble)
    if first && trouble == nil { return }
    let event = DriveEvent(at: Date(), kind: .note, text: "watch: \(trouble ?? "hears the guidance again")")
    Task { await log.append(event) }
  }

External links

Exercise

Add an append-only JSONL event log to a long-running program of yours (a server, a job, a game loop). Write each event the moment it happens, not at shutdown. Kill the process with kill -9 halfway through a run and confirm every line before the kill is readable. Then think of one failure you once debugged with nothing to go on, and add the single event that would have shown it.

Hint

Thin the high-frequency events (positions, ticks, frames) by time, and never thin the rare ones (errors, state changes); that is exactly the split Pathfinder makes between fixes and everything else. If your language buffers file writes, check whether a hard kill loses the buffer; open the file for appending and write whole lines. A kill -9 tests a crash, not a power cut: the kernel still holds what you wrote. Surviving a power cut takes an explicit sync (fsync, or FileHandle.synchronize() in Swift), and that costs a trip to storage on every call.

The Drive Log Is Evidence

A live experiment needs a lab notebook

What one drive writes

The road keeps finding things

When the log is silent, that is a finding too

Builds as the heartbeat

Code

External links

Exercise

Progress

Comments 0