Lesson 03 of 04 · published

Vendored Byte for Byte

~15 min · vendoring, swiftpm, provenance, c

Level 0Blank Canvas

0 XP0/37 lessons0/12 achievements

0/100 XP to next level100 XP to go0% complete

Four Targets, One Direction

Ariandel's engine is a Swift package, AriandelKit, and its four library targets stack in a single direction:

CJSONC: json-c, MIT, vendored at commit dcd1572. It's here for one reason: libmypaint parses .myb brush files with it.
CMyPaint: libmypaint, ISC, vendored at d5a88fb (master, 2026-04-14, the 2.0.0-beta API). It depends on CJSONC.
CAriandelSurface: Ariandel's own C. It's the tiled surface libmypaint paints into, plus the pixel kernels. It depends on CMyPaint.
AriandelEngine: Swift. Brushes, tiles, layers, documents, undo, files, rendering.

Two more targets ride along: ariandel-icon, a small executable that paints the app's icon with the engine itself, and the test target. The package targets iOS 18 and macOS 15, so swift test on a Mac is the fast engine suite. If SwiftPM targets are new to you, Swift Quest covers packages from the ground up.

Why Vendor at All

There were three ways to bring libmypaint in: a git submodule, a package fetched at build time, or the files copied into the repository. Two facts about the upstreams decided it.

Neither upstream ships a Swift package. libmypaint builds with autotools (configure.ac) and json-c with CMake. SwiftPM runs neither. Somebody has to run upstream's generators once (libmypaint's generate.py for two settings headers, json-c's CMake configured for iOS for three more), and those outputs have to live somewhere. A submodule can't hold extra files without going dirty, and there's no Swift package to fetch.
A shipping app should build from what's in its repository. With libmypaint's and json-c's bytes in the tree, the engine's C builds from the tree alone: no network fetch at build time, no upstream force-push, no moved tag. Every byte that compiles into the app shows up in the history, and an upstream update becomes one reviewable commit.

The cost is real: you own updates. The answer to that cost is a recipe written down beside the files, which comes up in a minute.

Three Kinds of File

"Vendored byte for byte" doesn't mean every file came from upstream untouched. It means every file is honestly one of three kinds, and a PROVENANCE.md in each vendored target says which.

	libmypaint (`CMyPaint`)	json-c (`CJSONC`)
Copied	The sixteen library sources and their private headers, `include/`, `fastapprox/`, and `glib/mypaint-brush.c` (compiled only through `mypaint-brush.c`'s `#include`)	Twelve library sources, their private headers, the public headers
Generated	`brushsettings-gen.h` and `include/mypaint-brush-settings-gen.h`, by upstream `generate.py` at the same commit	`json.h`, `json_config.h`, and `config.h`, by upstream CMake for iOS
Local	`config.h`, three defines	One guard inside the generated `config.h`: `HAVE_ENDIAN_H` only under `__has_include(<endian.h>)`, since the iOS SDK has that header and the macOS SDK doesn't

Note what the third row isn't: a fix to libmypaint's behavior. Local files configure a build. They never change what the engine does.

Proving It

"Unmodified" is only worth something if a machine can check it. The check is almost embarrassingly simple: for each vendored file, ask git for the same path at the recorded commit and cmp the two. One wrinkle: public headers sit in include/ here but at the upstream root, so the path has to be mapped back.

The sketch walks the whole vendored directory: every file at every depth except PROVENANCE.md itself, because a patch hides in a subdirectory as easily as at the top. Run against both targets on 2026-10-12, it found 51 identical libmypaint files (35 at the root and in include/, 15 under fastapprox/ and glib/, and COPYING) and 35 identical json-c files (34 plus COPYING). Each target had exactly three files the upstream commit doesn't contain, and they're exactly the generated and local files its PROVENANCE.md lists. The sketch makes both conditions part of its exit status. You pass it the files PROVENANCE.md declares, and it fails on any byte difference, on any other file the commit doesn't have, wherever in the tree it sits, and on a clone or commit that isn't there at all. That last guard matters more than it looks: without it, a wrong commit makes every file "not upstream", nothing differs, and the check passes. Running generate.py at d5a88fb reproduced both settings headers byte for byte. And git log agrees from a different angle: of 167 commits in the repository, exactly one has ever touched either vendored directory, the one that put them there.

The Rule and the Recipe

The libmypaint PROVENANCE.md ends with the whole policy in four sentences: "Never edit these files. A behavior libmypaint lacks belongs in Sources/CAriandelSurface or Swift. To update, copy the same file set from a newer upstream commit, regenerate the two headers, run swift test, and record the new commit here." json-c's adds one step: re-apply the HAVE_ENDIAN_H guard. That's the price of vendoring, paid in advance and written down where the next updater will trip over it.

A vendored file is either upstream's byte for byte, generated by upstream's own tool, or a declared local file that configures the build. Anything else is a fork that hasn't admitted it yet.

Code

Verbatim from Ariandel's Package.swift: the three C targets·swift

        // json-c, vendored byte-for-byte (see Sources/CJSONC/PROVENANCE.md).
        .target(
            name: "CJSONC",
            path: "Sources/CJSONC",
            exclude: ["COPYING", "PROVENANCE.md"],
            cSettings: [.headerSearchPath(".")]
        ),
        // libmypaint, vendored byte-for-byte (see Sources/CMyPaint/PROVENANCE.md).
        .target(
            name: "CMyPaint",
            dependencies: ["CJSONC"],
            path: "Sources/CMyPaint",
            exclude: [
                "COPYING",
                "PROVENANCE.md",
                // Compiled only through mypaint-brush.c's #include.
                "glib/mypaint-brush.c",
                "fastapprox/Makefile.am.local",
            ],
            cSettings: [
                .define("HAVE_CONFIG_H"),
                .headerSearchPath("."),
            ]
        ),
        // Ariandel's own C: the tiled surface libmypaint paints into, pixel kernels.
        .target(
            name: "CAriandelSurface",
            dependencies: ["CMyPaint"],
            path: "Sources/CAriandelSurface"
        ),

A minimal sketch: verify a vendored directory against the upstream commit its PROVENANCE.md records·bash

#!/bin/bash
# usage: verify-vendor.sh <upstream-clone> <commit> <vendored-dir> [declared-file ...]
# Walks every file under <vendored-dir>, at any depth, except PROVENANCE.md (the
# record itself). Each must equal the recorded commit's copy, byte for byte.
# A file the commit doesn't have passes only if it is declared on the command
# line: the generated and local files PROVENANCE.md lists. Anything else fails.
set -u
upstream=$1 commit=$2 vendored=$3
shift 3
declared=" $* "
# A wrong clone or commit must fail loudly, not report every file as "not upstream".
git -C "$upstream" rev-parse --verify --quiet "$commit^{commit}" >/dev/null 2>&1 ||
  { echo "no commit $commit in $upstream"; exit 2; }
same=0 differ=0 extra=0 undeclared=0
cd "$vendored" || exit 2
while IFS= read -r -d '' f; do
  f=${f#./}
  path=${f#include/}          # public headers sit at the upstream root
  if ! git -C "$upstream" cat-file -e "$commit:$path" 2>/dev/null; then
    extra=$((extra + 1))
    case "$declared" in
      *" $f "*) echo "declared: $f" ;;
      *) echo "UNDECLARED: $f"; undeclared=$((undeclared + 1)) ;;
    esac
    continue
  fi
  if git -C "$upstream" show "$commit:$path" | cmp -s - "$f"; then
    same=$((same + 1))
  else
    echo "DIFFERS: $f"; differ=$((differ + 1))
  fi
done < <(find . ! -type d ! -path ./PROVENANCE.md -print0 | sort -z)
echo "identical=$same differs=$differ not-upstream=$extra undeclared=$undeclared"
[ "$differ" -eq 0 ] && [ "$undeclared" -eq 0 ] && [ "$same" -gt 0 ]

What it printed for Ariandel's two vendored targets, then for a copy with one space appended to helpers.c, a copy with one undeclared file nested at glib/extra.c, and a commit that doesn't exist·text

$ verify-vendor.sh <libmypaint clone> d5a88fb Sources/CMyPaint \
    brushsettings-gen.h config.h include/mypaint-brush-settings-gen.h
declared: brushsettings-gen.h
declared: config.h
declared: include/mypaint-brush-settings-gen.h
identical=51 differs=0 not-upstream=3 undeclared=0

$ verify-vendor.sh <json-c clone> dcd1572 Sources/CJSONC \
    config.h include/json.h include/json_config.h
declared: config.h
declared: include/json.h
declared: include/json_config.h
identical=35 differs=0 not-upstream=3 undeclared=0

$ verify-vendor.sh <libmypaint clone> d5a88fb <copy with helpers.c touched> \
    brushsettings-gen.h config.h include/mypaint-brush-settings-gen.h
declared: brushsettings-gen.h
declared: config.h
DIFFERS: helpers.c
declared: include/mypaint-brush-settings-gen.h
identical=50 differs=1 not-upstream=3 undeclared=0        # exit status 1

$ verify-vendor.sh <libmypaint clone> d5a88fb <copy with an extra glib/extra.c> \
    brushsettings-gen.h config.h include/mypaint-brush-settings-gen.h
declared: brushsettings-gen.h
declared: config.h
UNDECLARED: glib/extra.c
declared: include/mypaint-brush-settings-gen.h
identical=51 differs=0 not-upstream=4 undeclared=1        # exit status 1

$ verify-vendor.sh <libmypaint clone> deadbeef Sources/CMyPaint
no commit deadbeef in <libmypaint clone>                  # exit status 2

External links

Exercise

Find a vendored directory in a project you work on (a third_party/, a vendor/, a copied single-header library). Work out which upstream commit it came from, and write that down in a provenance file beside it if nobody has. Then run the verify sketch against a clone at that commit, passing the generated and local files you declared. Classify every file it reports as copied, generated, or local, and for anything that's none of the three, decide whether it's a fork to declare or a patch to move out.

Hint

If nobody recorded the commit, git log on the vendored path usually shows when it arrived, and the upstream's history around that date narrows it down; the commit whose files produce the fewest DIFFERS lines is your best candidate. Once you have a clean run, turn the loop into a test or a CI step so the next edit gets caught on the day it's made.

Vendored Byte for Byte

Four Targets, One Direction

Why Vendor at All

Three Kinds of File

Proving It

The Rule and the Recipe

Code

External links

Exercise

Progress

Comments 0