Skip to content
C.W.K.
Stream
Lesson 05 of 05 · published

A Canvas Sized to the Device

~16 min · memory, metal, mipmaps, file-formats, ipados

Level 0Blank Canvas
0 XP0/37 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

The Canvas That Could Not Fit

On its first evening Ariandel let you make a 16384×16384 canvas on any iPad. Do the arithmetic the 6479f1f review did. That canvas is 268 million pixels. One layer's slice of the 8-bit GPU mirror is four bytes a pixel, exactly 1 GiB, and the layer array starts with at least four slices: 4 GiB before a single stroke. The rgba16Float composite adds 2 GiB more, plus a third again for its mipmaps. The tiles and the undo history come on top of that. The review's line was blunt: canvas size "allowed 16384x16384 regardless of device memory (GPU mirror alone would exceed RAM)."

The fix makes the canvas a function of the machine (first code block). CanvasSize.maximumPixels(memory:) reads the device's RAM, from ProcessInfo.processInfo.physicalMemory in the app, and answers 16 million pixels up to 4 GiB, 36 million up to 8 GiB, and 64 million above. Independently, no edge may exceed 16,384 pixels, the largest 2D texture Metal allows on every supported iPad. limited(toPixels:) scales a request down proportionally until both caps hold, rounding each side down.

The cap has a twin that is easy to miss: maximumLayers. Every layer costs one mirror slice, so the layer count is a budget of 1,400 MiB divided by the bytes one 8-bit slice takes, clamped between 4 and 200. A 16-megapixel canvas holds 22 layers. A 64-megapixel canvas holds 5. An A4 page at 300 dpi holds 42. The second code block runs the same rules for three device sizes so you can see the trade: a bigger canvas is a shallower stack, and Ariandel says so with a toast ("This canvas holds at most 5 layers") instead of failing later.

The cap is applied where a canvas is made or imported from another format, each door with its own policy, and two doors are not as strict as the rest:

  • New Canvas shows the result before you commit, for example "This iPad holds up to 36 megapixels, so the canvas will be 6000 × 6000" for a 16384 × 16384 request.
  • A photo or picture is decoded directly at the fitted size, so a huge image never sits in memory at full size first.
  • A PSD or PSB larger than the cap is scaled down whole, each pixel the average of the ones it covers, built a tile row at a time.
  • A .procreate file with more pixels than the cap is refused with its size and the limit, rather than resampled. Only the pixel count is checked there: a long strip under the pixel cap but past 16,384 on one edge is not refused at that door.
  • An .ariandel package, yours or one shared from another iPad, opens at the size it was saved. Nothing checks it against this iPad's cap, so a canvas made on an iPad with more memory arrives at its full size.

Two budgets stay fixed while the canvas cap moves: the layer mirror's 1,400 MiB and the undo history's 768 MiB (this track's first lesson). When memory runs short anyway, the warning path from that lesson sheds history and never touches layers.

Mipmaps, but Only When Smaller

Zoomed far out, a canvas pixel covers less than one screen pixel, and a plain bilinear sample skips most of the picture: fine lines flicker, hatching turns to moiré, and a pan makes the whole canvas shimmer. Mipmaps fix that with pre-shrunk copies the sampler blends between. The review added them, and the rule (third code block) is narrow: generate them only when transform.scale × contentScale < 1, that is, when one canvas pixel covers less than one device pixel, and only when the composite changed since the last build (every composite pass marks them stale). Then one blit, generateMipmaps(for:), before the display pass.

Watch the units. The zoom readout counts points, and an iPad's display has two device pixels per point, so this threshold is 50% on the readout, not 100%. The review's own record says "below 100 %"; the code says device pixels, and the code is right about where minification starts. Between 50% and 100% each canvas pixel still covers at least one device pixel, and mipmaps would only cost time.

Files Out

Every export is file-format work over the two readers from Track 3: tiles directly, or a flatten synchronized from tiles.

ExportWhat it is
PNG, transparent PNG, JPEGThe flattened picture.
TIFFLZW-compressed, tagged with the canvas DPI.
PDFA page sized from the canvas DPI (300 by default), so a 300 dpi A4 canvas is an A4 page; with Page Assist, one page per page.
PSD (layers)Layered, with the flattened picture as its composite.
PNG Files (each layer)Every layer as its own canvas-sized PNG, through its mask, straight from tiles.
Animated GIF, animated PNG, videoAnimation Assist's frames in playback order; GIF at 1080 px, video at 1920 px on the long edge.
Time-lapseThe recorded segments joined into one MP4, in full or condensed to thirty seconds.

The time-lapse records through AVAssetWriter into fragmented movies at 1080p, 2K or 4K. The codec is chosen by frame size: H.264 up to a 1920-pixel long edge, HEVC past it, because H.264's largest frames cannot hold every canvas (a 4K square is beyond them) and HEVC carries the detail in fewer bits. The bit rate follows the pixel count: three bits per pixel per second for H.264, about 6 Mb/s at 1080p, and two for HEVC.

Files In

Ariandel reads other apps' formats as files. It is not affiliated with their makers, and it reads only what a file contains.

  • PSD and PSB: RGB or grayscale, 8 or 16 bits, raw, PackBits or ZIP channels, with names, opacity, blend modes, clipping and masks. Folders become groups one level deep; adjustment layers have no pixels and are left out, and counted.
  • .procreate artworks: a zip holding Document.archive, a keyed archive, and one folder of LZ4-compressed tiles per layer. Only the entries the layers need are read, from a memory-mapped file. Older files whose tiles are LZO are refused.
  • Brushes: MyPaint .myb presets (alone or zipped into a pack, previews alongside), Procreate's .brush and .brushset, and Photoshop's .abr, versions 6 to 10. What maps onto the engine is carried over; what does not (wet mix, color dynamics, most other dynamics) is left out, never guessed.

The keyed archive is the instructive one. NSKeyedUnarchiver would rebuild the archive's objects by instantiating the classes it names, and those classes belong to another app. KeyedArchive (third code block) never does that: it parses the property list and walks $objects by reference index, so strings, numbers and dates come out as values and everything else stays a dictionary. No code from the file runs, and a hostile file is just a strange plist.

Size the work to the machine, and say so up front. A limit derived from the device and shown before the user commits (the canvas size, the layer count) is a design choice. The same limit discovered as a crash halfway through a painting is a bug.

Code

The canvas cap, its layer twin, and the fit (AriandelEngine/Artwork.swift, CanvasSize, verbatim)·swift

    /// The largest edge any canvas may have (Metal's texture limit on every
    /// supported iPad).
    public static let maximumEdge = 16384

    /// The most pixels a canvas may hold on a device with `memory` bytes of
    /// RAM. Every layer keeps a GPU mirror and the history keeps tiles, so
    /// the canvas grows with the device: 16 MP up to 4 GB, 36 MP up to
    /// 8 GB, 64 MP above.
    public static func maximumPixels(memory: UInt64) -> Int {
        let gigabyte: UInt64 = 1 << 30
        if memory <= 4 * gigabyte { return 16_000_000 }
        if memory <= 8 * gigabyte { return 36_000_000 }
        return 64_000_000
    }

    /// The most layers a canvas this size holds: every layer keeps a GPU
    /// mirror, and the history needs room beside them.
    public var maximumLayers: Int {
        let budget = 1_400 * 1024 * 1024
        return max(4, min(200, budget / max(1, pixelCount * 4)))
    }

    /// This size scaled down, keeping its proportions, until it fits both the
    /// edge limit and `maximumPixels`.
    public func limited(toPixels maximumPixels: Int) -> CanvasSize {
        var scale = 1.0
        let longest = Double(max(width, height))
        if longest > Double(Self.maximumEdge) {
            scale = Double(Self.maximumEdge) / longest
        }
        let pixels = Double(width) * Double(height) * scale * scale
        if pixels > Double(maximumPixels) {
            scale *= (Double(maximumPixels) / pixels).squareRoot()
        }
        guard scale < 1 else { return self }
        return CanvasSize(width: max(1, Int((Double(width) * scale).rounded(.down))),
                          height: max(1, Int((Double(height) * scale).rounded(.down))))
    }
A minimal sketch: the same caps for 4, 8 and 16 GB devices (compiled and run with swiftc in Swift 6 mode)·swift
// A minimal sketch of Ariandel's canvas cap: the same rules as CanvasSize's
// maximumPixels, limited(toPixels:) and maximumLayers, run for three RAM sizes.
// Build and run: swiftc -swift-version 6 canvas.swift -o canvas && ./canvas
import Foundation

let maximumEdge = 16384                       // Metal's 2D texture limit on supported iPads

func maximumPixels(memory: UInt64) -> Int {
    let gigabyte: UInt64 = 1 << 30
    if memory <= 4 * gigabyte { return 16_000_000 }
    if memory <= 8 * gigabyte { return 36_000_000 }
    return 64_000_000
}

func limited(_ width: Int, _ height: Int, toPixels cap: Int) -> (Int, Int) {
    var scale = 1.0
    let longest = Double(max(width, height))
    if longest > Double(maximumEdge) { scale = Double(maximumEdge) / longest }
    let pixels = Double(width) * Double(height) * scale * scale
    if pixels > Double(cap) { scale *= (Double(cap) / pixels).squareRoot() }
    guard scale < 1 else { return (width, height) }
    return (max(1, Int((Double(width) * scale).rounded(.down))), max(1, Int((Double(height) * scale).rounded(.down))))
}

func maximumLayers(_ width: Int, _ height: Int) -> Int {
    let budget = 1_400 * 1024 * 1024          // bytes of 8-bit RGBA mirror the layers may take
    return max(4, min(200, budget / max(1, width * height * 4)))
}

let requests = [(16384, 16384), (4961, 7016), (2480, 3508), (20000, 500)]   // A4 at 600 and at 300 dpi
for gb: UInt64 in [4, 8, 16] {
    let cap = maximumPixels(memory: gb << 30)
    print("\(gb) GB device: cap \(cap / 1_000_000) MP")
    for (w, h) in requests {
        let (fw, fh) = limited(w, h, toPixels: cap)
        print("  \(w)x\(h) -> \(fw)x\(fh)  (\(fw * fh / 1000) k px, up to \(maximumLayers(fw, fh)) layers)")
    }
}
print("this Mac:", ProcessInfo.processInfo.physicalMemory >> 30, "GB ->",
      maximumPixels(memory: ProcessInfo.processInfo.physicalMemory) / 1_000_000, "MP")
Three format and display decisions (three files, verbatim excerpts)·swift
// AriandelEngine/CanvasRenderer.swift: mipmaps only when the canvas is shown smaller than its pixels
    public func encodeDisplay(artwork: Artwork, options: DisplayOptions, pass: MTLRenderPassDescriptor,
                              commandBuffer: MTLCommandBuffer) {
        guard let composite else { return }
        let screenScale = options.transform.scale * options.contentScale
        if screenScale < 1, mipmapsStale, composite.mipmapLevelCount > 1,
           let blit = commandBuffer.makeBlitCommandEncoder() {
            blit.generateMipmaps(for: composite)
            blit.endEncoding()
            mipmapsStale = false
        }
        guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else { return }
        // ...

// AriandelEngine/KeyedArchive.swift: reading Procreate's archives without running their classes
/// Reading a keyed archive (NSKeyedArchiver's plist) without its classes, as
/// Procreate's brushes and documents are kept: objects are walked by their
/// references, never decoded, so no code from the file runs. Strings,
/// numbers and dates come out as themselves; anything else stays a plist
/// dictionary to look into.
struct KeyedArchive {
    let objects: [Any]
    let root: [String: Any]

    init?(_ data: Data) {
        guard let plist = try? PropertyListSerialization.propertyList(from: data, format: nil) as? [String: Any],
              let objects = plist["$objects"] as? [Any],
              let top = plist["$top"] as? [String: Any],
              let index = top["root"].flatMap(Self.uid), index < objects.count,
              let root = objects[index] as? [String: Any] else { return nil }
        self.objects = objects
        self.root = root
    }
    // ...

// AriandelMobile, Support/Timelapse.swift: HEVC only past 1080p
        /// Past 1080p, H.264's largest frames no longer hold every canvas (a 4K
        /// square is beyond it), and HEVC carries the detail at half the bits.
        var codec: AVVideoCodecType { max(width, height) > Timelapse.longEdge ? .hevc : .h264 }

        /// Bits a second for the codec at this size: 6 Mb/s at 1080p for H.264.
        var bitRate: Int {
            let pixels = Double(width * height)
            return max(2_000_000, Int(pixels * (codec == .hevc ? 2 : 3)))
        }

External links

Exercise

Run the sketch, then add your own devices: read physicalMemory on your Mac and look up your phone's or tablet's RAM. For each, compute the GPU memory a full-size canvas would take before any painting: four 8-bit slices plus an rgba16Float composite with mipmaps. Compare it with the device's RAM. Then pick an app you use that has a size limit (a canvas, an upload, a document) and find out whether it tells you the limit before you hit it.
Hint
Four slices at 4 bytes a pixel is 16 bytes a pixel; the composite is 8 bytes a pixel, and a full mip chain adds about a third (8 × 4/3 ≈ 10.7). So budget roughly 27 bytes per pixel before the first stroke: about 1.7 GB for a 64-megapixel canvas, and over 7 GB for an uncapped 16384×16384.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.