Skip to content
C.W.K.
Stream
Lesson 08 of 09 · published

Ink Stays Markdown

~13 min · pencil-notes, pencilkit, png-chunk, source-model, judgment, v3

Level 0Cold Draft
0 XP0/43 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete
"The drawing can live in the note without the note learning to be anything but text."

The Feature That Didn't Fit the Decoration Shape

Track 4 ended with a test: you would know Rekindle had outgrown its core when a feature arrived that did not reduce to "compute ranges over source, paint a layer, mutate on accept." On 2026-10-01 one arrived. Dad asked for pencil notes on the iPad on the level of Apple Notes: "펜슬을 사용을 해서 그림도 그릴 수 있고 … OCR를 해서 텍스트를 추출할 수 있고, 그런데 맥에서 보면, 리킨들에서 보면 그림으로 나타날 수 있고" — draw or write with the Pencil, pull the words out by recognition, and see it in Rekindle on the Mac as a picture. A drawing is not a range of characters. The question was whether the source model would have to bend.

It didn't. The answer was not a new substrate; it was to put the non-text thing where non-text things already live — in an asset — and keep the note itself plain Markdown.

One Asset Carries Both

A drawing is one content-named PNG in assets/: the page rendered in the light appearance on white paper, at the page's full width with a margin around the ink. Inside that same file rides PencilKit's own editable drawing, in a private ancillary PNG chunk named rkIN, next to a small JSON header (schema 1, engine pencilkit, page width, crop, scale, the recognized text, and who read it). It is the editable-PNG shape Excalidraw and draw.io use. The chunk name is not decoration either: in PNG's naming rules, a lowercase first letter means ancillary and a lowercase second letter means private, so every reader that does not know the chunk is allowed to skip it. The Mac, the site and Finder all see an ordinary picture. Sync, the asset store and the Mac app did not change at all. Identical drawing and text give identical bytes, so content naming still holds.

The note holds an ordinary image reference as its own paragraph, and under it the recognized words as a block quote, line for line. The quote lives in the note's source, so the Mac's and the device's search both find it and copying takes it. It sits beside the drawing, never instead of it. A drawing with no words gets no quote.

Rekindle on an iPad in landscape: a note titled Pencil Note holding a pencil drawing of a circle with an arrow on white paper, with a blue pen badge in its corner; below it the word Pippa handwritten in the note body. The library sidebar on the left shows Pinned notes, status filters and smart folders, and its footer reads Syncing with office…; a pencil button floats above the New Note button.
The pencil drawing is a picture in assets with its editable ink inside; the pen badge reopens it. The handwritten word below is ordinary text going into the note.
Rekindle v5.0.0 on the Mac with pencil-note-2.md open: the same circle-and-arrow drawing shown as a large picture in the editor, the word Pippa as a plain line of text beneath it, and the status bar reading Synced 66 change(s).
The same note after it synced to office: the Mac sees an ordinary picture and a line of Markdown. Nothing about the Mac changed to show it.

The two screenshots above are the same note. On the iPad, the drawing (a circle and an arrow, no words, so no quote) wears the pen badge that marks it as a pencil drawing — a tap on it reopens the page to continue. Below it, the word Pippa is ordinary handwriting written straight into the note's text, which the system turns into typed characters. On the Mac, after the sync line reads Synced 66 change(s)., the drawing is just a picture and Pippa is a plain line of text. The note file on disk is a frontmatter block, one image reference to ../assets/<hash>.png, and that line.

Pippa Reads the Page; the Device Stands In

The first build recognized handwriting on the device, with Vision, Korean first and then English. On Dad's real pen strokes it missed — the one handwritten English word in that night's test was read neither by the iPad nor by macOS Vision over the same strokes at any crop or scale. Dad chose the other option the same night: "2번으로 가자. 피파가 읽게 해줘." (Let's go with option 2. Let Pippa read it.)

So on Done the page is read twice. The device's Vision reading runs locally. And on a live replica, office Pippa reads the same page through the macro lane the device already uses for rewrites (POST /api/admin/utility-macros/apply-free), carrying the picture as a JPEG bounded at 2048 px, through the central editor_pippa slot's brain and model, with no persona and a transcription instruction that answers NONE for a picture without writing. Pippa's answer wins. The device's reading stands when she is unreachable, has no PIN, errors, stays silent for 60 seconds, or is skipped from the page. Either way the chunk header records reader: pippa or reader: device. Measured that night over loopback with the exact request the device sends, Pippa read the word correctly in 5.7 s, answered NONE for a drawing in 3.9 s, and read three lines of Korean and English lettering in 2.6 s.

Continuing a Drawing

Reopen a pencil note and draw more, and Done files a new picture beside the old one (assets only ever add) and moves the reference to it. The quote follows the new words only while it still reads exactly what the last version wrote. A quote Dad corrected or deleted by hand is his, and stays — he is told. Erasing everything saves nothing and keeps the page open. A drawing from a newer PencilKit than the device has, or with a damaged chunk, says so and keeps its picture. The Mac shows the ink and never edits it; that is the first version's scope by Dad's choice, and the feature stays app-owned until it proves itself, before anything moves into the shared kit.

Code

The note stays plain Markdown — a picture reference, then the words as a quote·markdown
---
title: "Pencil Note"
tags: []
---

![Pencil note](../assets/<content-hash>.png)

> the words the page was read as,
> line for line

(A drawing with no words gets no quote. The editable ink rides
inside the PNG's private rkIN chunk, not in this file.)
Two readers, one answer recorded with its source (sketch of the device's compose step)·swift
// On Done: render the page, read it on the device, and ask office Pippa.
let page = render(drawing)                        // light appearance, white paper
let device = await visionText(page.png)           // ko-KR, then en-US
var pippa: String? = nil
if let reader {                                   // a live replica with a PIN
  pippa = await pippaReading(page.png, reader: reader, skip: skip)
  // nil when unreachable, on error, after 60 s of silence, or on Skip
}
let text = pippa ?? device
let by = (pippa == nil) ? "device" : "pippa"
let payload = InkPayload(header: .init(pageWidth: page.width, crop: page.crop,
                                       scale: page.scale, text: text, reader: by),
                         drawing: drawing.dataRepresentation()) // editable strokes
let png = try InkPNG.embed(payload, in: page.png) // the rkIN chunk

External links

Exercise

Pick a kind of content your own notes cannot hold today — a sketch, a voice memo, a table you edit in another app. Design how it would live beside a plain-text note without changing the note format: which file carries it, what the note keeps (a reference, a summary, searchable words), and how the editable original survives a round trip through tools that do not understand it.
Hint
Look for a container format with an official place for private data that other readers skip — PNG ancillary chunks, PDF attachments, image metadata. The note then needs only a reference and whatever text you want search to find.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.