Coalesced Yes, Predicted No

The Pencil Is Faster Than the Screen

An Apple Pencil on an iPad Pro is sampled up to 240 times a second. UIKit does not wake your view 240 times a second. It delivers a touch event at its own pace and folds the samples in between into that one event. Apple's documentation is blunt about what a drawing app should do next: call coalescedTouches(for:) on the event, and you get every sample reported since the last delivery, in order, the last one a copy of the touch you were handed. Skip that call and you paint with only the last sample of each event, a polygon where the hand drew a curve.

Ariandel's canvas view says its policy in its first lines: "every Pencil event's coalesced touches (240 Hz on iPad Pro) become StylusInputs in canvas space and go to the painter in one batch per event, so a frame renders exactly what the hand did." The code is the feed function in the first code block. It takes the coalesced samples (or the touch alone when there is no event to ask), converts each through lesson one's single door, keeps the last one aside as the stroke's lift point, and hands the whole array to continueStroke in one call. The Drawing Pad does the same with its moving touches, and so does a hand-drawn selection, which extends its outline by every coalesced sample.

One Event, One Batch, One Redraw

"One batch per event" is a structural choice, not just a performance one. Inside Painter.continueStroke, the samples are fed to libmypaint between surface.begin() and surface.end(), one atomic batch. libmypaint queues the dabs, the surface renders them into tiles at end() (Track 2), and only then do Ariandel's own per-batch steps run: the clone and dual-brush merges, the grain, and keeping the paint inside a selection (Track 5). Coalescing therefore sets the grain of all of that work at one pass per delivered event, never one per sample.

The display side is honest about its own rate. The canvas is an MTKView with preferredFramesPerSecond = 120, but it is paused with enableSetNeedsDisplay on, so it draws on demand. Every continueStroke ends by asking for a display, and the commit that built the canvas (d24155e) describes it as "redrawing on demand at up to 120 Hz". The Pencil samples at twice the screen's best rate, and the batch is what reconciles the two.

Predicted Touches: Zero Hits, on Purpose

UIKit offers a second array on the same event, predictedTouches(for:): where the system guesses the Pencil will be next, so an app can draw a little ahead of the hand and hide some latency. Search Ariandel's sources for predicted and nothing touch-related comes back. The only near miss is the Photoshop reader undoing ZIP "prediction", which is a compression term.

Absence of evidence is not a reason by itself, so here is the reason. A prediction is a draft that has to be thrown away a few milliseconds later. An app whose strokes are vector paths can afford that: draw the guess into a throwaway overlay, drop it next frame. Ariandel's strokes are not paths. Every input goes into libmypaint, which commits dabs into tiles immediately and also moves its own state forward: the smoothed position, the fraction of a dab already travelled, the smudge buckets, the random input. Erasing the pixels a guess painted would not be enough. The second code block is a compiled toy with just two pieces of that state. Feed it one predicted point, erase every dab the prediction laid, then feed the real points. The stroke still comes out different, with fewer dabs, and two of them land off the line. Taking a prediction back honestly would mean snapshotting and restoring both the tiles and the brush state every frame, or running a second engine just for the guess.

So Ariandel pays the latency instead. Is that latency felt? Nobody knows yet. The engine evaluation that chose libmypaint lists "latency under Apple Pencil" under what was not checked, and as of 2026-10-12 not one of the 22 coop history rows for cwkAriandel mentions latency. That is an open question, not a settled trade.

Estimated Properties: Taken as Delivered, Refinements Ignored

There is a third stream. For some samples UIKit cannot report final force, altitude or azimuth in time, so it sends estimates and later calls touchesEstimatedPropertiesUpdated(_:) with refined values, matched up by an estimationUpdateIndex. A vector app can patch the stored point. Ariandel keeps the estimate itself: the sample carrying it goes into the batch like any other, with whatever force and tilt UIKit had at the time. What it drops is the correction. It overrides the method with an empty body and a comment that states the policy: "Force and tilt arrive refined a few milliseconds later; strokes use the coalesced values as delivered, which is what libmypaint expects." By the time a refinement lands, the dab it would have changed is already in a tile, and the brush state has moved past it. Same constraint as prediction, same answer: an engine that commits takes its input once, in order.

Take every input you can keep, and none you would have to take back. Coalesced samples are real history and cost nothing to keep, estimated values included, taken as first delivered. A predicted touch is a draft, and a refinement is an edit to a past that is already painted. A pipeline that commits on contact can accept either only by building a whole second, revocable path beside it.

Code

Canvas/CanvasMetalView.swift, verbatim: refinements ignored on purpose, and one batch of coalesced samples per event·swift

    override func touchesEstimatedPropertiesUpdated(_ touches: Set<UITouch>) {
        // Force and tilt arrive refined a few milliseconds later; strokes use
        // the coalesced values as delivered, which is what libmypaint expects.
    }

    private func feed(_ touch: UITouch, event: UIEvent?) {
        guard let model else { return }
        let samples = event?.coalescedTouches(for: touch) ?? [touch]
        let inputs = samples.map { input(from: $0, model: model) }
        lastStrokeInput = inputs.last
        model.continueStroke(inputs)
    }

A minimal sketch (compiled with swiftc and run): erase every dab a predicted point laid and the stroke is still not the same, because the brush's state moved·swift

import Foundation
import simd

// A minimal sketch of why a dab-committing engine cannot take back a predicted
// touch: the pixels are only half of it. Like libmypaint, this toy brush keeps
// state between inputs (a smoothed position and a fraction of a dab owed),
// and every input it is fed moves that state for good.
struct ToyBrush {
    var position: SIMD2<Double>? = nil
    var owed = 0.0                 // fraction of the next dab already travelled
    let spacing = 4.0, smoothing = 0.5
    var dabs: [SIMD2<Double>] = []

    mutating func stroke(to target: SIMD2<Double>) {
        guard let from = position else { position = target; return }
        let to = from + (target - from) * smoothing          // slow tracking
        let distance = simd_length(to - from)
        var travelled = (1 - owed) * spacing
        while travelled <= distance {
            dabs.append(from + (to - from) * (travelled / distance))
            travelled += spacing
        }
        owed = 1 - (travelled - distance) / spacing
        position = to
    }
}

let real: [SIMD2<Double>] = [[0, 0], [10, 0], [20, 0], [30, 0], [40, 0]]
let predicted: SIMD2<Double> = [34, 9]                     // the guess for the next frame

var honest = ToyBrush()
for p in real { honest.stroke(to: p) }

var guessed = ToyBrush()
for p in real.prefix(3) { guessed.stroke(to: p) }
let committed = guessed.dabs.count
guessed.stroke(to: predicted)                              // a frame drawn early
guessed.dabs.removeSubrange(committed...)                  // "erase" those dabs perfectly
for p in real.suffix(2) { guessed.stroke(to: p) }          // the real touches arrive

func show(_ dabs: [SIMD2<Double>]) -> String {
    dabs.map { String(format: "(%.1f,%.1f)", $0.x, $0.y) }.joined(separator: " ")
}
print("real only:            ", show(honest.dabs))
print("predicted, then erased:", show(guessed.dabs))
print("same stroke:", honest.dabs == guessed.dabs)

// Prints:
// real only:             (4.0,0.0) (8.0,0.0) (12.0,0.0) (16.0,0.0) (20.0,0.0) (24.0,0.0) (28.0,0.0)
// predicted, then erased: (4.0,0.0) (8.0,0.0) (12.0,0.0) (26.5,2.4) (30.4,1.6)
// same stroke: false

Exercise

Take any pipeline of yours that consumes a stream and commits as it goes: a drawing view, a game's physics step, an event-sourced log, a streaming parser. List every piece of state a single input moves forward, not just the visible output. Then decide, for that pipeline, what it would cost to accept a speculative input and later retract it, and write the answer down as either a snapshot you would need or a second path you would need to build.

Hint

The visible output is rarely the expensive part. Look for accumulators: running averages, partial counts carried to the next step, random number generators, caches keyed on the last value. Each one is something a retraction must restore, and each one is invisible in a screenshot.

The Pencil Is Faster Than the Screen

One Event, One Batch, One Redraw

Predicted Touches: Zero Hits, on Purpose

Estimated Properties: Taken as Delivered, Refinements Ignored

Code

External links

Exercise

Progress

Comments 0