~16 min · metal, gpu, compositing, blend-modes, shaders
Level 0Blank Canvas
0 XP0/37 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
A Shader in a Swift String
Ariandel's GPU code is one file, ShaderSource.swift, and that file is one Swift raw string literal of Metal source. CanvasRenderer compiles it when it starts, with device.makeLibrary(source:options:), and pulls its functions out by name. The file's first line says why: "compiled at runtime so the package needs no shader build step." The engine is a Swift package of C and Swift targets that runs its whole test suite on a Mac with swift test; keeping the shaders as a string means no .metal files to compile, no .metallib to bundle, and no Xcode-only step between editing a blend formula and testing it. The price is a compile at startup (the app keeps one shared renderer, so it pays once per launch), and errors that surface at runtime as RendererError.shaderCompilation instead of at build time. The engine suite constructs renderers on the Mac's GPU, so a broken shader still fails the suite before it can reach an iPad.
A string has one more trick a .metal file does not: interpolation. The blend switch inside the shader takes its case numbers from Swift, case \#(LayerBlendMode.multiply.shaderIndex):, and shaderIndex is simply the mode's position in allCases (third code block). Files store the mode's name, never the number, so the enum's order can change freely and the shader can never disagree with it. The buffers the shader reads are declared once in a C header, ariandel_shader_types.h, so Swift sees C's layout; the Metal source repeats them field for field, and a test pins every size and offset that matters (a compose layer is 40 bytes, the uniforms 112).
Every Layer Is a Slice
Track 3 met the mirror: every layer, and every mask's paint, is one slice of a single texture2DArray in rgba8Unorm, premultiplied, with private storage. The renderer keeps a slot map from layer id to slice. A layer that leaves gives its slot back for reuse; the array starts at max(4, layers + 2) slices and grows by four, copying the old slices across with a blit. Only dirty tiles are uploaded. One texture for the whole stack means one binding: the compositor reaches any layer by index, which is exactly what a loop over a stack wants.
One Kernel, One Thread per Pixel
ard_compose (first code block) is a compute kernel. Each thread owns one pixel of the region being redrawn, starts from the background (or transparent), and walks the stack bottom to top. For each layer it does, in order:
Read the layer's pixel from its slice.
Floating picture. If a Transform is in progress on this layer, map the pixel through a 3×3 projective matrix into the lifted picture, divide by w (it is 1 for every affine placement), sample, and lay it over the layer's own pixels. The preview is therefore the committed result, computed before opacity and blend.
Mask. Scale by the mask paint's luminance, plus the unpainted value where nothing is painted: white shows, black hides. A mask is paint read as luminance, so it can be painted with any brush.
Tint, for Animation Assist's onion skins.
Opacity, then clipping: a clipped layer is multiplied by the alpha of the last unclipped layer below it, opacity included, which is how a clipping group behaves.
Blend with ard_blend_over.
The blend function is the W3C Compositing and Blending Level 1 formula written for premultiplied color: src·(1−αb) + dst·(1−αs) + αs·αb·B(cb, cs), where B is the mode's function on un-premultiplied colors. Normal skips the division entirely. There are 26 modes: the W3C separable and non-separable set (multiply, screen, overlay, soft light, hue, color and the rest) plus painters' extras such as linear burn, vivid, linear and pin light, hard mix, add, subtract, divide, and darker and lighter color.
The kernel writes into a float target, never back into a slice. For the screen that target is the rgba16Float composite, and the kernel runs over the changed region only: the uniforms carry an origin and a size, and the dispatch covers just that rectangle. The region is the union of the tiles uploaded this frame; a change to the stack itself (order, visibility, opacity, blend, clipping, mask, background) recomposites the whole canvas. The second code block is a sketch of this shape you can run on any Mac: a string compiled at runtime, two layers in an rgba8Unorm array, a normal pass over the whole canvas, then a multiply pass over a 4×4 region, and a pixel outside that region still holding the first result.
One Triangle to the Glass
Display is a second, tiny pipeline. ard_display_vertex emits a single oversized triangle that covers the screen; the fragment shader maps each screen pixel back to the canvas through the view transform's 3×3 matrix (zoom, rotation, position, mirror), samples the composite, and adds the workspace color outside the canvas, a checkerboard under transparency, the pixel grid when zoomed in far enough, and the selection's stripes and marching ants. Sampling turns to nearest-neighbor once a canvas pixel covers two or more screen pixels, so close-up paint stays crisp.
Be precise about speed. The canvas view sets preferredFramesPerSecond = 120, but it also sets isPaused = true and enableSetNeedsDisplay = true, which in MetalKit's own terms makes it event-driven: the internal draw loop that frame rate governs is paused, and the view draws when the model asks it to. Each draw synchronizes dirty tiles, composites the changed region, and presents. No measurement of how a heavy canvas holds up on Dad's iPad is on record.
One formula, one home. The display, every flattened export, Pippa's canvas look and merge down all composite through this same kernel. When a blend formula lives in exactly one place, fixing it once fixes every composited picture the app makes, and no two of them can quietly disagree. Exports that keep the layers apart, PNG Files and the layers inside a PSD, composite nothing: they read each layer straight from its tiles on the CPU, PNG Files applying the mask with a C helper and the PSD carrying it as a mask channel (lesson five).
Code
The blend and the compose kernel (AriandelEngine/ShaderSource.swift, inside the Metal string, verbatim)·metal
/// Composites premultiplied `src` over premultiplied `dst` with blend `mode`.
static inline float4 ard_blend_over(float4 dst, float4 src, int mode) {
float as = src.a;
float ab = dst.a;
if (as <= 0.0) { return dst; }
if (mode == \#(LayerBlendMode.normal.shaderIndex) || ab <= 0.0) { return src + dst * (1.0 - as); }
float3 cs = src.rgb / as;
float3 cb = dst.rgb / ab;
float3 mixed = clamp(ard_blend_color(mode, cb, cs), 0.0, 1.0);
float3 rgb = src.rgb * (1.0 - ab) + dst.rgb * (1.0 - as) + as * ab * mixed;
return float4(rgb, as + ab * (1.0 - as));
}
/// Every visible layer over the background, bottom to top. A clipping mask
/// is multiplied by the alpha of the last unclipped layer (its opacity
/// included), Photoshop's clipping group. The floating picture of a
/// transform in progress is drawn over its layer's own pixels before that
/// layer's opacity and blend apply, so the preview is the committed result.
/// A layer mask scales the layer by its paint's luminance over the
/// unpainted value (white shows, black hides), before opacity and blend,
/// so a clipping base's masked-out parts clip too; the mask paint moving
/// with a floating picture lies over the mask's own first, as commit draws it.
kernel void ard_compose(texture2d_array<float, access::read> layers [[texture(0)]],
texture2d<float, access::write> output [[texture(1)]],
texture2d<float, access::sample> floating [[texture(2)]],
texture2d<float, access::sample> floatingMask [[texture(3)]],
constant ComposeLayer *stack [[buffer(0)]],
constant ComposeUniforms &u [[buffer(1)]],
sampler floatingSampler [[sampler(0)]],
uint2 gid [[thread_position_in_grid]]) {
if (gid.x >= u.size.x || gid.y >= u.size.y) { return; }
uint2 p = gid + u.origin;
float4 dst = u.hasBackground != 0 ? u.background : float4(0.0);
float baseAlpha = 1.0;
for (uint i = 0; i < u.layerCount; i++) {
float4 content = layers.read(p, stack[i].slice);
bool placed = false;
float2 floatingAt = float2(0.0);
if (int(i) == u.floatingIndex) {
// Projective: a distorted picture divides by w (1 for every affine placement).
float3 source = u.floatingMatrix * float3(float2(p) + 0.5, 1.0);
if (source.z > 1e-6) {
placed = true;
floatingAt = source.xy / source.z / u.floatingSize;
float4 f = floating.sample(floatingSampler, floatingAt);
content = f + content * (1.0 - f.a);
}
}
if (stack[i].mask >= 0) {
float4 m = layers.read(p, uint(stack[i].mask));
if (placed && u.floatingHasMask != 0) {
float4 fm = floatingMask.sample(floatingSampler, floatingAt);
m = fm + m * (1.0 - fm.a);
}
content *= clamp(ard_lum(m.rgb) + stack[i].maskUnpainted * (1.0 - m.a), 0.0, 1.0);
}
if (stack[i].tintAmount > 0.0) {
float3 tint = float3(stack[i].tintRed, stack[i].tintGreen, stack[i].tintBlue) * content.a;
content.rgb = mix(content.rgb, tint, stack[i].tintAmount);
}
float4 src = content * stack[i].opacity;
if (stack[i].clip != 0) {
src *= baseAlpha;
} else {
baseAlpha = src.a;
}
dst = ard_blend_over(dst, src, stack[i].blend);
}
output.write(dst, p);
}
A minimal sketch: a runtime-compiled kernel over an rgba8Unorm layer array, redrawing one region (compiled and run with swiftc in Swift 6 mode on an Apple silicon Mac)·swift
// A minimal sketch of Ariandel's compositor shape: Metal source as a Swift string,
// compiled at runtime; layers as slices of one rgba8Unorm 2D array texture; one
// compute kernel that blends the stack into a float target over a region only.
// Build and run on a Mac: swiftc -swift-version 6 compose.swift -o compose && ./compose
import Metal
let normal: Int32 = 0, multiply: Int32 = 1
let source = """
#include <metal_stdlib>
using namespace metal;
struct Layer { uint slice; float opacity; int blend; };
struct Uniforms { uint2 origin; uint2 size; uint count; };
static inline float4 blend_over(float4 dst, float4 src, int mode) {
if (src.a <= 0.0) { return dst; }
if (mode == \(normal) || dst.a <= 0.0) { return src + dst * (1.0 - src.a); }
float3 cs = src.rgb / src.a, cb = dst.rgb / dst.a;
float3 mixed = cb * cs; // \(multiply): multiply
float3 rgb = src.rgb * (1.0 - dst.a) + dst.rgb * (1.0 - src.a) + src.a * dst.a * mixed;
return float4(rgb, src.a + dst.a * (1.0 - src.a));
}
kernel void compose(texture2d_array<float, access::read> layers [[texture(0)]],
texture2d<float, access::write> output [[texture(1)]],
constant Layer *stack [[buffer(0)]],
constant Uniforms &u [[buffer(1)]],
uint2 gid [[thread_position_in_grid]]) {
if (gid.x >= u.size.x || gid.y >= u.size.y) { return; }
uint2 p = gid + u.origin; // only the changed region
float4 dst = float4(1.0); // white paper
for (uint i = 0; i < u.count; i++) {
dst = blend_over(dst, layers.read(p, stack[i].slice) * stack[i].opacity, stack[i].blend);
}
output.write(dst, p);
}
"""
struct Layer { var slice: UInt32; var opacity: Float; var blend: Int32 }
struct Uniforms { var origin: SIMD2<UInt32>; var size: SIMD2<UInt32>; var count: UInt32 }
let device = MTLCreateSystemDefaultDevice()!
let library = try device.makeLibrary(source: source, options: nil) // no .metal file, no build step
let pipeline = try device.makeComputePipelineState(function: library.makeFunction(name: "compose")!)
let width = 8, height = 8
let arrayDescriptor = MTLTextureDescriptor()
arrayDescriptor.textureType = .type2DArray
arrayDescriptor.pixelFormat = .rgba8Unorm
arrayDescriptor.width = width
arrayDescriptor.height = height
arrayDescriptor.arrayLength = 2
arrayDescriptor.usage = [.shaderRead]
arrayDescriptor.storageMode = .shared
let layers = device.makeTexture(descriptor: arrayDescriptor)!
// Slice 0: opaque orange. Slice 1: half-transparent blue, premultiplied.
@MainActor func fill(_ slice: Int, _ rgba: [UInt8]) {
let pixels = [UInt8]((0..<(width * height)).flatMap { _ in rgba })
layers.replace(region: MTLRegionMake2D(0, 0, width, height), mipmapLevel: 0, slice: slice,
withBytes: pixels, bytesPerRow: width * 4, bytesPerImage: width * height * 4)
}
fill(0, [255, 128, 0, 255])
fill(1, [0, 0, 128, 128])
let target = device.makeTexture(descriptor: {
let d = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: .rgba32Float, width: width, height: height,
mipmapped: false)
d.usage = [.shaderRead, .shaderWrite]
d.storageMode = .shared
return d
}())!
@MainActor func pixel(_ x: Int, _ y: Int) -> SIMD4<Float> {
var value = SIMD4<Float>(repeating: 0)
target.getBytes(&value, bytesPerRow: 16, from: MTLRegionMake2D(x, y, 1, 1), mipmapLevel: 0)
return value
}
@MainActor func run(_ stack: [Layer], region: (x: Int, y: Int, w: Int, h: Int)) {
var uniforms = Uniforms(origin: SIMD2(UInt32(region.x), UInt32(region.y)),
size: SIMD2(UInt32(region.w), UInt32(region.h)), count: UInt32(stack.count))
let queue = device.makeCommandQueue()!
let buffer = queue.makeCommandBuffer()!
let encoder = buffer.makeComputeCommandEncoder()!
encoder.setComputePipelineState(pipeline)
encoder.setTexture(layers, index: 0)
encoder.setTexture(target, index: 1)
stack.withUnsafeBytes { encoder.setBytes($0.baseAddress!, length: $0.count, index: 0) }
encoder.setBytes(&uniforms, length: MemoryLayout<Uniforms>.stride, index: 1)
encoder.dispatchThreads(MTLSize(width: region.w, height: region.h, depth: 1),
threadsPerThreadgroup: MTLSize(width: 8, height: 8, depth: 1))
encoder.endEncoding()
buffer.commit()
buffer.waitUntilCompleted()
}
func show(_ p: SIMD4<Float>) -> String { "(\(p.x), \(p.y), \(p.z), \(p.w))" }
run([Layer(slice: 0, opacity: 1, blend: normal), Layer(slice: 1, opacity: 1, blend: normal)], region: (0, 0, 8, 8))
print("whole canvas, normal: ", show(pixel(0, 0)))
// The blue layer turns to multiply; only the 4x4 region it changed is recomposited.
run([Layer(slice: 0, opacity: 1, blend: normal), Layer(slice: 1, opacity: 1, blend: multiply)], region: (2, 2, 4, 4))
print("inside region, multiply:", show(pixel(3, 3)))
print("outside, untouched: ", show(pixel(0, 0)))
Compiled at runtime, numbered by the enum, pinned by a test (four files, verbatim excerpts)·swift
// AriandelEngine/ShaderSource.swift
/// Metal source for Ariandel's compositor and canvas display, compiled at
/// runtime so the package needs no shader build step.
///
/// The blend formulas are the W3C Compositing and Blending Level 1 set plus
/// the painters' extras (linear burn, darker and lighter color, add, vivid,
/// linear and pin light, hard mix, subtract, divide). This file is their only
/// home: display, export, Pippa's canvas look and merge all composite here,
/// and so do clipping masks, layer masks and the floating picture of a
/// transform in progress. Case numbers are interpolated from
/// `LayerBlendMode.shaderIndex`, so the enum's order can change freely.
enum ShaderSource {
static let metal = #"""
#include <metal_stdlib>
// ...
case \#(LayerBlendMode.multiply.shaderIndex): return b * s;
case \#(LayerBlendMode.darken.shaderIndex): return min(b, s);
// ...
// AriandelEngine/Layer.swift
/// Index the compositor's shader switches on. Stored nowhere (files keep
/// the raw value); the shader source takes its case numbers from here.
public var shaderIndex: Int32 {
Int32(LayerBlendMode.allCases.firstIndex(of: self)!)
}
// AriandelEngine/CanvasRenderer.swift (init)
let library: MTLLibrary
do {
library = try device.makeLibrary(source: ShaderSource.metal, options: nil)
} catch {
throw RendererError.shaderCompilation(String(describing: error))
}
// Tests/AriandelEngineTests/RendererTests.swift
@Test func bufferLayoutsMatchTheMetalSource() {
#expect(MemoryLayout<ArdComposeLayer>.stride == 40)
#expect(MemoryLayout<ArdComposeLayer>.offset(of: \.tintAmount) == 36)
#expect(MemoryLayout<ArdComposeLayer>.offset(of: \.mask) == 16)
#expect(MemoryLayout<ArdComposeUniforms>.stride == 112)
#expect(MemoryLayout<ArdComposeUniforms>.offset(of: \.floatingIndex) == 24)
// ...
}
Run the sketch, then add a third blend mode: screen, which W3C defines as B(cb, cs) = cb + cs − cb·cs. Give it the number 2, interpolate it into the shader the way the sketch interpolates normal, and run the region pass with it. Predict the inside pixel before you run it. Then swap the numbers of multiply and screen in Swift only and confirm the output still matches, because the shader took its numbers from the same constants.
Hint
Add one line right after the multiply line: if (mode == \(screen)) { mixed = cb + cs - cb * cs; }, so multiply stays the fallback. With an orange base (1, 0.502, 0) under pure blue (0, 0, 1) at alpha 0.502, screen's mixed color is (1, 0.502, 1), and the formula gives 0.498 × base + 0.502 × mixed: about (1.0, 0.502, 0.502, 1).
Progress
Progress is local-only — sign in to sync across devices.