The iOS Host: Bridge and Custom Scheme

The same bundle, in a native frame

The iPhone and iPad app does not reimplement the studio. A build script compiles the web client and copies it into the app's Web/ folder with a file naming the commit it came from. The app, built with XcodeGen for iOS 18 on iPhone and iPad in Swift 6, hosts that bundle in a WKWebView and adds what a browser cannot do well: picking original photos, sharing files, reading offline, and playing clips through its own scheme. The shared plumbing comes from the family kit (CwkKitCore, CwkKitWeb, CwkKitMedia), which Sibling Kit Quest explains.

A packaged origin with one door

The page loads from avatars-app://client, served from the bundle by a custom scheme handler. The navigation policy allows exactly one kind of main-frame load: the app's own origin at / or /index.html. A link to Pippa's panel opens in a separate sheet with its own web view, and that remote Sidekick never receives the native bridge. Firebrand Quest's lesson "One Locked WebView, Not a Browser" makes the same split for a coding client: one locked view for the work, a second isolated view for family chrome.

Script talks to native code through one message handler, and the kit's origin-gated handler admits a message only if six things hold: it came from this web view, from the main frame, from the page's content world, from a sender whose scheme and host are the app's own, while the web view itself is still on that origin, and within a size budget. For Avatars the budget is 145 << 20 bytes, room for a 100 MB photo after base64.

The bridge replaces fetch

bridge.js is injected at document start and does nothing unless it runs in the top frame at avatars-app://client. There it swaps window.fetch for a version that sends every same-origin /api/ call through the message handler. Form fields travel as text, files as base64, with the same 100 MB limit the engine enforces. Anything else goes to the browser's own fetch untouched.

Why not let the scheme handler take API writes too? Because WKWebView can discard the body of a POST sent to a custom scheme. So the scheme serves only GET resources and the bundle, and every write rides the gated message, as a comment in StudioHost.swift says. The bridge also keeps a long timer of 965 seconds, and its message is a rule in disguise: the request is still pending, check saved batches before generating again. A slow answer is never a reason to submit twice.

One API policy on the native side

Engine is the only place that turns a page's path into a network request. The path must start with /api/, carry no host or scheme, no backslash or fragment, no encoded slash or backslash, and no .. segment, and the resolved URL must land on the studio's own host and port. A test feeds it a full foreign URL, a protocol-relative one, /internal/providers, two kinds of dot-dot escape and both encoded slashes, and every one is refused. The family session token lives in the Keychain, this device only, and rides an X-Pippa-Session header. A 401 or 403 retires the offline reads saved under that credential.

Sharing is narrower still. A click on a link to an original or a delivery export is intercepted, the host fetches the file, names it by its real type, and hands it to the share sheet. A clip is shared as .mp4, never as an image. No other route can be shared.

175 requests to the first frame

Clips broke the simple path. WebKit's player reads a video through the scheme one byte range at a time and waits for each read before asking the next: a two-byte probe, box headers eight bytes at a time, then the index at the end of the file in reads of four to eight bytes, then sample by sample. It plays only from 206 answers with exactly the requested bytes and a Content-Range that places them, and on iPhone it goes full screen unless the configuration allows inline playback.

Relaying each range to the engine worked, slowly. Every engine request paid about 50 ms of authorization before its first byte, so an eight-second 720p clip took 175 requests and 6.2 seconds to its first frame, measured on the engine host's own loopback before any network round trip. ClipBlocks reads the engine in 1 MiB blocks instead, each one placed by the engine's own Content-Range, keeps up to 64 MiB, reads the next block ahead, and lets waiters for one block share a single read. The player's ranges are cut from those blocks. The same clip took 4 engine reads and played in real time, 8.08 seconds of wall time for 8 seconds of video. An original never changes, so a kept block stays true, and every block is still read with the engine's authorization.

Principle: When a client reads in tiny sequential steps, never let each step pay the full price of a trip.

Code

cwkAvatars, clients/apple/AvatarsMobile/Resources/bridge.js: fetch for same-origin /api/ calls goes through the native message handler (verbatim, lines 34-53)·javascript

  const browserFetch = window.fetch.bind(window);
  window.fetch = async (input, init = {}) => {
    const u = new URL(typeof input === 'string' ? input : input.url, location.href);
    if (u.protocol !== location.protocol || u.host !== location.host || !u.pathname.startsWith('/api/')) return browserFetch(input, init);
    const values = {path:u.pathname+u.search, method:init.method || 'GET'};
    if (init.body instanceof FormData) {
      values.fields = [];
      for (const [name, value] of init.body.entries()) {
        if (typeof value === 'string') values.fields.push({name, value});
        else {
          if (value.size > 100*1024*1024) throw new Error('Choose an image up to 100 MB.');
          values.fields.push({name, data:await base64(value)});
        }
      }
    } else if (init.body != null) values.body = String(init.body);
    // `onUploadProgress` (not a fetch option; the page's upload passes it) hears
    // an original's bytes as native sends them (cwkSiblingKit todo #545).
    const reply = await call('fetch', values, init.onUploadProgress);
    return new Response(reply.body, {status:reply.status, headers:{'Content-Type':'application/json'}});
  };

cwkAvatars, clients/apple/AvatarsMobile/Sources/Engine.swift: the only path policy between the page and the network (verbatim, lines 46-55)·swift

    static func url(_ path: String) throws -> URL {
        guard path.hasPrefix("/api/"), !path.contains("\\"), !path.contains("#"),
              let parts = URLComponents(string: path), parts.host == nil, parts.scheme == nil,
              !parts.percentEncodedPath.lowercased().contains("%2f"),
              !parts.percentEncodedPath.lowercased().contains("%5c"),
              !parts.path.split(separator: "/").contains(".."),
              let result = URL(string: path, relativeTo: origin)?.absoluteURL,
              result.host == origin.host, result.port == origin.port else { throw BridgeError.invalidPath }
        return result
    }

cwkAvatars, clients/apple/AvatarsMobile/Sources/StudioHost.swift: one player range cut from engine blocks and answered 206 (verbatim, lines 255-278)·swift

    private func relay(_ range: ByteRange, of url: URL, to schemeTask: WKURLSchemeTask) async throws {
        await blocks.validateScope()
        let path = url.path + (url.query.map { "?" + $0 } ?? "")
        let size = blocks.size
        if blocks.file(path) == nil {
            // The first block read tells the clip's length.
            if case .from(let first, _) = range { _ = try await blocks.block(path, first / size) } else { _ = try await blocks.block(path, 0) }
        }
        guard let file = blocks.file(path), let span = range.span(in: file.total),
              let head = MediaRange.partial(url: url, span: span, total: file.total, type: file.type) else { throw BridgeError.badReply }
        try Task.checkCancellation()
        schemeTask.didReceive(head)
        for index in (span.lowerBound / size)...(span.upperBound / size) {
            let block = try await blocks.block(path, index)
            blocks.prefetch(path, index + 1)
            try Task.checkCancellation()
            let start = index * size
            let lower = max(span.lowerBound, start) - start, upper = min(span.upperBound, start + block.data.count - 1) - start
            guard lower <= upper else { throw BridgeError.badReply }
            schemeTask.didReceive(block.data.subdata(in: lower..<(upper + 1)))
        }
        try Task.checkCancellation()
        schemeTask.didFinish()
    }

Exercise

Build a small iOS app with a WKWebView that loads a page from a custom scheme and contains a video tag whose source is also on that scheme. In your WKURLSchemeHandler, log every Range header the player sends for a short MP4 whose index sits at the end of the file. First answer with the whole file in a plain URLResponse and observe what happens, then answer each range with a 206 HTTPURLResponse carrying Content-Range and Content-Length. Count the reads before the first frame. Finally, serve the ranges from 1 MiB blocks cached in memory and count again.

Hint

Set allowsInlineMediaPlayback on the configuration, or iPhone will take the video full screen. To get an index at the end, write the clip with AVAssetWriter and leave shouldOptimizeForNetworkUse off. Keep a dictionary of running tasks so the stop callback can cancel the right one.

The same bundle, in a native frame

A packaged origin with one door

The bridge replaces fetch

One API policy on the native side

175 requests to the first frame

Code

External links

Exercise

Progress

Comments 0