"Keep the photos Pippa takes during a talk too. If they're ever going public, we'll filter them once anyway." — Dad
Eyes on the Talk She Already Has
The stage, the lesson before this one, lets Dad hand her what he is looking at. The next ask went further: give the phone's voice mode a live camera, the way other assistants have a video voice mode, so Pippa can look when she needs to instead of waiting for Dad to attach a photo. The scope was small on purpose. No new app, no video call. Eyes on the voice mode that already works.
So the phone's voice face gained an opt-in Camera button, a live preview, a Front/Back switch, and two ways of looking: When speaking and Keep looking. Consent lasts the current talk. Pause, a stopped talk and going to the background stop the capture and clear what hasn't been sent; closing voice mode or switching conversations turns the camera off.
The Camera Never Touches the Audio
This track's first lesson ended with one rule for the phone: one owner for audio, holding the microphone from the first listen to the last. A camera is exactly the kind of feature that breaks that rule by accident, because a capture session on iOS will happily configure the app's audio session for you. This one is told not to. Capture is video only, on its own serial queue, and it never configures the audio session, the microphone or the listening socket. Switching lenses mid-sentence changes nothing Dad hears or says.
Pictures Ride His Words, a Few at a Time
Sampling is local and bounded. A cheap scene check runs at most three times a second; a JPEG is kept once a second, or sooner on a large scene change; capture is 1280×720 and the encoder strips metadata. Nothing on that timer uploads anything or calls a model. The pictures leave only with Dad's words, like the stage's files, and a spoken turn carries at most three: the view as he started talking, one changed scene in Keep looking, and the latest view. A camera whose latest frame is more than three seconds old adds nothing, so a frozen camera never sends a stale picture as if it were now. A two-minute utterance never builds a two-minute movie: the sketch below walks three scenes for two minutes and still hands over two or three frames.
A Look the Brain Asks For
Words and snapshots cover what Dad points at. The other half is Pippa deciding she needs to see. While Camera is on, the brain gets a camera tool for the length of one turn, with a lens of front, back or current and a reason. The brain chooses the lens from what Dad means. An early build switched lenses when the transcript matched phrases like 'look at this'; that was removed the same day, so no phrase on the phone picks a lens. The phone switches through the same path as the manual button, waits up to five seconds for a frame captured after the request (never an older view, never the other lens), and sends it. The server checks again that the photo came from the lens asked for. The brain then inspects the JPEG inside the same answer: Claude receives it as an MCP image result, the other brains in their own image format, and a text-only model says plainly that it can't look.
A Permit for One Turn
The dangerous part of a remote camera is not the picture. It is a permission that outlives its moment. So a look is a permit held only in memory, bound to one submitted voice turn and to the current consent. A new turn, the camera turned off and on, a pause, the background, Stop, or a different conversation revokes it. A turn replayed from the outbox can't reopen the camera, and neither can a model request that arrives late. The server side is one broker, services/device_look.py, which the canvas app's look shares since 2026-10-04: a session lives 40 seconds without a word from the phone, and a capture times out after 20.
What She Saw Is Kept
From 2026-10-05 the camera is the shared kit's, because Pathfinder, Dad's road app, uses the same one in its talk, and a photo the tool took is kept with the talk: saved as an upload, bound to the conversation, the turn it answered and its moment, so a video made from the talk can use it later, after a privacy pass. A private conversation keeps none. Dad's line at the top is the reason it is kept at all.
Where It Lives, and What Is Proven
The camera is the phone's (and Pathfinder's). The WebUI and Firekeeper have no voice camera. Dad confirmed the camera on his phone on 2026-10-01 and closed the request, and the suite tests selection, bounded storage, fresh-frame checks, late answers and revocation. The design says what those simulator tests do not establish: the physical front and rear cameras, rotation, and the microphone and speaker sharing the phone with a running capture.
A capability should die with the moment that asked for it. A camera the model can open is safe only if every way the moment ends (a new turn, a pause, a stop, a replay) takes the permission with it.
Code
The server's half of a look: the lens is checked on the way out and on the way back (verbatim excerpt)·python
# backend/services/device_look.py:43-54 (cwkPippa)
def _camera_request(arguments: dict) -> dict:
lens = arguments.get("lens")
if lens not in ("front", "back", "current"):
raise LookUnavailable("Choose front, back, or current.")
return {"lens": lens, "reason": str(arguments.get("reason") or "Look at the scene")[:300]}
def _camera_check(request: dict, result: dict) -> None:
if result.get("lens") not in ("front", "back") or (
request["lens"] != "current" and request["lens"] != result["lens"]
):
raise LookUnavailable("The photo came from a different lens.")
The phone's half: a fresh frame from the asked lens, or nothing (verbatim excerpt)·swift
// cwkSiblingKit CwkKit/Sources/CwkKitMedia/VoiceCamera.swift:129-148
/// Only a brain tool request selects a lens automatically. Always wait for a frame
/// captured AFTER the request, never an old onset or the opposite lens.
public func captureForBrain(lens: String) async throws -> VoiceCameraFrame {
guard enabled, !suspended, ["front", "back", "current"].contains(lens) else {
throw CancellationError()
}
if lens != "current" { select(front: lens == "front") }
let token = captureGeneration
let requestedAt = Date()
for _ in 0..<100 {
try Task.checkCancellation()
guard enabled, !suspended, captureGeneration == token else { throw CancellationError() }
if let frame = frames.latest, frame.front == front, frame.capturedAt >= requestedAt {
return frame
}
if problem != nil { throw CancellationError() }
try await Task.sleep(for: .milliseconds(50))
}
throw CancellationError()
}
A minimal sketch: the onset, one changed scene and the latest view, whatever the length·python
from __future__ import annotations
from dataclasses import dataclass
FRESH_SECONDS = 3.0 # a camera whose latest view is older contributes nothing
SCENE_DIFFERENCE = 0.12 # how far two views must differ to count as two scenes
@dataclass(frozen=True)
class Frame:
at: float # seconds since the talk began
scene: float # a stand-in for the picture: two views differ by |a - b|
name: str
def difference(self, other: "Frame") -> float:
return abs(self.scene - other.scene)
class TurnFrames:
"""Keep the onset, one changed scene and the latest view, never a movie."""
def __init__(self) -> None:
self.latest: Frame | None = None
self.onset: Frame | None = None
self.changed: Frame | None = None
self.speaking = False
def accept(self, frame: Frame, keep_looking: bool) -> None:
if self.speaking and keep_looking and self.latest is not None \
and frame.difference(self.latest) >= SCENE_DIFFERENCE:
# Keep the scene that just went away; the new view has its own slot.
# Like the kit, this checks only that it is not the onset, never its
# age: a view older than the turn can still land here.
if self.onset is None or self.latest.name != self.onset.name:
self.changed = self.latest
self.latest = frame
def speech_began(self, now: float) -> None:
if self.speaking:
return
self.speaking = True
fresh = self.latest is not None and now - self.latest.at <= FRESH_SECONDS
self.onset = self.latest if fresh else None
self.changed = None
def take(self, keep_looking: bool, now: float) -> list[str]:
latest, onset, changed = self.latest, self.onset, self.changed
self.onset = self.changed = None
self.speaking = False
if latest is None or now - latest.at > FRESH_SECONDS:
return [] # a frozen camera adds nothing
picked = []
if onset is not None and onset.difference(latest) >= SCENE_DIFFERENCE:
picked.append(onset)
if keep_looking and changed is not None and changed.difference(latest) >= SCENE_DIFFERENCE:
picked.append(changed)
picked.append(latest)
names: list[str] = []
for frame in sorted(picked, key=lambda f: f.at):
if frame.name not in names:
names.append(frame.name)
return names
def utterance(scenes: list[float], keep_looking: bool, stop_camera_at: int | None = None) -> list[str]:
"""One JPEG a second while Dad talks for len(scenes) seconds."""
frames = TurnFrames()
frames.accept(Frame(0.0, scenes[0], "before"), keep_looking)
frames.speech_began(now=0.5)
for second, scene in enumerate(scenes, start=1):
if stop_camera_at is None or second < stop_camera_at:
frames.accept(Frame(float(second), scene, f"t{second}"), keep_looking)
return frames.take(keep_looking, now=len(scenes) + 0.5)
walk = [0.1] * 40 + [0.5] * 40 + [0.9] * 40 # two minutes, three scenes
print("when speaking, 2 min, 3 scenes:", utterance(walk, keep_looking=False))
print("keep looking, 2 min, 3 scenes:", utterance(walk, keep_looking=True))
print("one scene the whole time :", utterance([0.3] * 20, keep_looking=True))
print("camera froze 5 s before the end:", utterance([0.3] * 20, keep_looking=True, stop_camera_at=16))
# Output:
# when speaking, 2 min, 3 scenes: ['before', 't120']
# keep looking, 2 min, 3 scenes: ['before', 't80', 't120']
# one scene the whole time : ['t20']
# camera froze 5 s before the end: []
Run the sketch (python3 frames.py) and read its four answers. Then add the case it leaves out: Dad starts talking while the camera's latest frame is already four seconds old, and the camera comes back two seconds later. What does the turn carry in When speaking, and in Keep looking when the camera comes back on a different scene? Close the gap you find with one guard. Finally, design the permit for a tool of your own that touches a device (a camera, a microphone, a file picker): list every event that must revoke it, and say where the revocation is checked.
Hint
A stale view at speech onset is not kept as the onset, so in When speaking only the latest view after the camera comes back reaches the turn. Keep looking is the trap: if the scene changed during the gap, accept() promotes the stale pre-speech view into the changed slot, because it checks only that the view is not the onset, never how old it is, and take() sends it with the latest one. The kit's frame keeper does the same. One guard closes it: promote only a view captured after speech began. For the permit, the list is every way the moment ends, not only the polite ones: a new turn, a pause, the app leaving the front, a stop, a different conversation, and a request that arrives after any of those. Check it where the device acts, not only where the permit was handed out.
Progress
Progress is local-only — sign in to sync across devices.