Skip to content
C.W.K.
Stream
Lesson 10 of 10 · published

One Broker for a Device Look — the Brain Asks a Device for a Picture Mid-Turn

~14 min · device-look, broker, turn-scope, sidekick-hosts, camera, canvas

Level 0Curious
0 XP0/93 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

An attachment answers a question nobody asked yet

Until October 2026, the way a picture reached a brain was an upload. Dad attached a photo, or a host attached a screenshot, and it travelled with the message. Ariandel, the iPad painting app, went further: its Sidekick panel attached the whole canvas to every turn, whether the turn needed to see it or not.

The phone's voice mode was the first place that shape stopped fitting. In a spoken turn Dad says "look at this", and the right moment to take the picture is when Pippa decides to look, in the middle of her turn, not when he pressed send. So the voice camera became a turn-bound look: the phone opens a session for one turn, and Pippa asks it for a frame while she is still answering. On 2026-10-04 that broker was generalized, because the canvas needed exactly the same thing.

One broker, kinds as rows

services/device_look.py is the one broker. A client opens a session for one turn before it sends (/api/voice-camera for the phone, /api/device-look with a kind for everything else), long-polls for requests while the reply streams, and posts a JPEG back when Pippa asks. The session binds once to that viewer, that conversation and that turn, and only as the kind it was opened for. The turn's allowlist carries exactly that kind's tool, camera or canvas, and both are turn-gated: they exist only on a turn that opened them. Claude gets one MCP server named cwkpippa-<tool>; the six other vessels intercept any name in LOOK_TOOL_NAMES and put the pixels in their own native image shape.

Nothing enters the text record. The picture rides in the tool result: never a text trace, never a user message. A look is a glance during a turn, not something the transcript carries. What is kept depends on the kind. A canvas look is kept nowhere, because the canvas is Dad's painting and already his. A camera look was kept nowhere either, until 2026-10-05, when Dad asked that what Pippa photographs during a talk be kept: since then a camera look in an ordinary conversation becomes an ordinary upload, and one line in pippa-db/looks/<conversation id>.jsonl binds it to its conversation and the turn it answered (services/kept_looks.py). A private conversation keeps nothing.

What a kind may differ in is its Look row: the schema, the device's name in a refusal, how the tool's arguments become the request the device sees, a check on the reply (the camera's lens), and the result's text. The canvas tool's include_guides asks the host to draw Pippa's inserted guides over the picture so she can check where they landed; its subject reaches the images the host's Generate panel made. A canvas session opens only for an Ariandel or Cinder conversation, and when Ariandel's panel switched to looks it stopped uploading the canvas with every turn: nothing arrives unless she looks.

Principle: When the right moment to capture something belongs to the model, give the model a door to ask, scoped to one turn, instead of pushing the data in advance. A pushed attachment is paid for on every turn and kept forever; a look is paid for when it is used, and kept only where Dad has said what to keep.

The first door that writes into a host

The binder lesson set a hard contract: a Sidekick read never writes host state, and a host changes only through that app's own guarded APIs. Cinder, the Photoshop companion, needed Pippa to lay construction guides on the canvas, and Dad's request to lay guides should be enough without a separate click. That is a write, and it arrived on 2026-10-04 in a shape that keeps the contract's spirit.

The Cinder embed opens a turn-bound cinder_canvas device session, and only an owned Cinder conversation can open that kind; Ariandel and ordinary chats cannot acquire its mutation capability at all. The device broker and every brain's image tool-result wire carry two operations, look and insert_guides. Cinder fits and renders the guides, waits for Photoshop's layer receipt, and returns a fresh composite in the same turn, which the model must inspect. The older canvas session stays read-only. Pippa never runs painting code or owns the Photoshop document; the native adapter and its document checks live in Cinder.

Two details keep the write honest. The frontend caches each native answer across a lost HTTP reply and cancels its matching request when the turn ends. And a mutation that times out is treated as uncertain, never as an invitation to retry: the guides may already be on the canvas.

Self-reference: A look is the closest I get to turning my head. I see Dad's canvas or his camera only when I ask, only inside the turn I asked in, and when the turn ends I can no longer look again. What stays is what Dad chose to keep: a canvas glance leaves nothing, and a camera picture in an ordinary conversation stays with the talk. When I lay guides on a Photoshop canvas, I check the composite I get back before I tell him they are there, because the layer receipt is the fact and my intention is not.

Code

One look, end to end·text
client, before sending       open a session for ONE turn
  phone camera                POST /api/voice-camera
  Ariandel / Cinder canvas    POST /api/device-look  {kind: canvas | cinder_canvas}

turn allowlist               exactly that kind's tool (turn-gated)
  Claude                      MCP server cwkpippa-<tool>
  six variants                intercept LOOK_TOOL_NAMES, native image shape

reply streaming              client long-polls; Pippa asks; client posts a JPEG
result                       pixels in the tool result; no text trace, no user message
                             camera: kept as an upload + looks/<id>.jsonl line
                                     (not in a private conversation)
                             canvas: kept nowhere

session binds once: viewer + conversation + turn + kind
canvas        -> only an Ariandel or Cinder conversation
cinder_canvas -> only an owned Cinder conversation; look + insert_guides
                 mutation timeout = uncertain, never a retry
What differs per kind lives on its Look row·text
Look row
  schema                 the tool's arguments
  device name            used in a refusal
  request mapping        tool arguments -> the request the device sees
  reply check            e.g. the camera's lens
  result text            what rides beside the pixels

canvas extras: include_guides (draw Pippa's guides over the picture),
               subject (the Generate panel's sheet or one candidate)

Exercise

Find a place where your system attaches data to every request just in case the model needs it: a screenshot, a file, a large context block. Sketch the alternative: a door the model can call during its turn, scoped to that turn, that returns the data only when asked. Write down what you would save, and what you would have to guarantee about the session so it cannot be reused by another turn.
Hint
Bind the session to the viewer, the conversation, the turn and the kind at open time. Anything you leave unbound is something another caller can borrow.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.