The Translated Picture

The last and slowest layer

Spark's rule is fastest first. A photo moment shows Dad the lines the phone located, with my translations drawn over them as my text streams in. That overlay is quick and exact, but it is boxes laid on a photo. The last layer in the design is the photo itself redrawn, with every line replaced by its translation where it stands. It is an Ember job, it comes last, and it comes only when asked.

On the phone the button reads High-res picture. It appears on a photo moment once I have read its lines, while no translated picture exists and none is being drawn. After an attempt that did not finish, the same button reads Redraw picture. The engine enforces the same order from its side: asking for a picture before any line has been read is refused with a 400.

Spark writes the recipe

Ember is the family's image engine, and it knows nothing about translation. The first code block is why it does not need to. Spark writes the edit itself: the target language by name, then every line numbered, original and translation side by side, and a short list of constraints. Same place, a size that fits the same space, the original lettering's weight, colour and style as closely as the new script allows, everything else untouched, no new text, and any text not listed left alone.

The lines are the moment's line results that carry both a source and a translation. If a line was revised, the revision takes its original's place, exactly as every other view of the moment shows it. So the picture never disagrees with what Dad has already read on the screen. The translation stays mine; the picture is Ember drawing it in. How Ember chooses a generator and runs it is Ember & Cinder Quest's subject.

Which photo Ember sees

The reference sent to Ember is the moment's for-pippa copy when there is one: the downscaled photo, 2048 pixels on its long edge, that I read and numbered. Only when it is missing does the engine take any other image on the moment. It goes up inside the request as a base64 data URL.

The shape is the kit's choice, in the third code block. Ember's generators offer a handful of landscape ratios, and the kit picks the closest. A portrait photo always gets 1:1, the only ratio that does not squash it. Most menus are photographed upright, so most translated menus come back square. The architecture calls this the high-resolution layer, and the size is in fact Ember's call: Spark passes no size, so the picture comes back at the generator's default.

A job that survives a restart

Drawing takes a while, and an engine can restart in the middle. The second code block is built around that. It first looks for a picture-job result belonging to this turn. Only when there is none does it compose the recipe, submit the job, and immediately record a picture-job result carrying Ember's job id and the number of lines. That record is the whole trick. If the engine dies while waiting, recovery runs the same turn again, finds the recorded job id, and goes back to waiting on the same job. A test restarts with a job already recorded and asserts that nothing was submitted and the wait was for that job: never asked twice. One small window remains open, and it is worth seeing: a crash after Ember accepts the job but before the record is written would submit once more on recovery.

The waiting itself is the kit's. It polls every three seconds for up to thirty minutes. A 404 on a poll means Ember restarted and lost the job, since Ember keeps job records in memory, so the wait ends with that reason instead of treating silence as still running.

When the job is done, the image is fetched and stored as the moment's translated-picture media. Only then does Spark release Ember's hold. Ember protects an exploration result until the app that asked acknowledges a durable copy. Spark's attribution rode in with the job: the purpose exploration, source_app set to cwkSpark, and the moment and turn ids. Ember's cross-app exploration browser filters by that attribution, and its per-app retention keys on source_app. If the release fails, the image simply stays protected in Ember, which is the safe side. The turn finishes recording the media id, Ember's image id, and whether the release went through.

The flow has a family history worth knowing. cwkEasel ran this submit, wait, fetch and release sequence first. Spark's picture was the second copy, so the generic half moved into the Sibling Kit and both apps now vendor it. Sibling Kit Quest teaches that second-occurrence rule.

On the screen

While the job runs, the phone says that Ember is redrawing the photo, with the time elapsed. A failure shows its reason. When the picture arrives, a segmented control switches between Translated picture and Overlay; the web has a Translated and Original toggle. The commit that built it records a check on a test engine with a thirteen-line menu: about one minute end to end, layout and lettering kept.

Principle: For any job you do not control, write its id down the moment it is accepted, before you start waiting. A restart should resume a wait, never repeat a request.

Code

cwkSpark, backend/pictures.py: the edit recipe Spark writes for Ember (verbatim, lines 33-48)·python

def picture_prompt(lines: list[dict[str, Any]], target: str) -> str:
    """The edit recipe: the same photo, each line's words replaced by its
    translation where the line stands, nothing else changed."""
    language = language_name(target) or target
    listed = "\n".join(
        f"{line.get('n', index)}. “{line['source']}” → “{line['translation']}”"
        for index, line in enumerate(lines, start=1)
    )
    return (
        f"Edit this photo so its printed text reads in {language}. Replace each line of text below with its "
        "translation, in the same place, at a size that fits the same space, matching the original lettering's "
        "weight, color and style as closely as the new script allows. Keep everything else exactly as it is: "
        "the layout, the background, the objects, the light, the framing. Do not add text that is not listed, "
        "and leave any text not listed untouched.\n\n"
        f"Lines (original → {language}):\n{listed}"
    )

cwkSpark, backend/pictures.py: the job id is recorded before the wait, and the hold released after the copy (verbatim, lines 77-128)·python

            job_id = next(
                (item.get("job_id") for item in moment.get("results") or []
                 if item.get("kind") == "picture-job" and item.get("turn_id") == turn["id"]),
                None,
            )
            if job_id is None:
                lines = self._picture_lines(moment)
                if not lines:
                    await self._fail(turn, "Pippa has not read any lines of this photo yet")  # type: ignore[attr-defined]
                    return
                photo = next((m for m in moment.get("media") or [] if m.get("role") == "for-pippa"), None) or next(
                    (m for m in moment.get("media") or [] if str(m.get("content_type") or "").startswith("image/")),
                    None,
                )
                path = photo and corpus.media_path(moment["id"], photo["media_id"], photo["filename"])
                if not path:
                    await self._fail(turn, "the photo has not arrived")  # type: ignore[attr-defined]
                    return
                from .turns import _pixel_size

                size = await asyncio.to_thread(_pixel_size, path)
                payload = await asyncio.to_thread(path.read_bytes)
                job_id = await self.ember.submit(
                    prompt=picture_prompt(lines, moment.get("target_language") or "ko"),
                    references=[image_data_url(payload, photo.get("content_type") or "image/jpeg")],
                    source_metadata={
                        "purpose": "exploration",
                        "source_app": SOURCE_APP,
                        "moment_id": moment["id"],
                        "turn_id": turn["id"],
                    },
                    aspect_ratio=closest_aspect_ratio(*size) if size else None,
                )
                await self._write(  # type: ignore[attr-defined]
                    corpus.add_result, moment["id"], turn_id=turn["id"],
                    target_language=moment.get("target_language") or "ko",
                    fields={"kind": "picture-job", "job_id": job_id, "lines": len(lines)},
                )
            record = await self.ember.wait(job_id)
            image = await self.ember.fetch(record)
            extension = "png" if image.content_type.endswith("png") else "jpg"
            media = await self._write(  # type: ignore[attr-defined]
                corpus.add_media, moment["id"], new_ulid(),
                filename=f"translated.{extension}", content_type=image.content_type,
                role=PICTURE_ROLE, payload=image.data,
            )
            released = await self.ember.release(image.image_id)
            await self._write(  # type: ignore[attr-defined]
                corpus.finish_turn, turn["id"], "completed",
                media_id=(media or {}).get("media_id"), ember_image_id=image.image_id,
                ember_released=released, results=1,
            )

cwkSpark, backend/kit_ember_jobs.py, vendored from cwkSiblingKit: a portrait photo takes 1:1 (verbatim, lines 66-74)·python

def closest_aspect_ratio(width: int, height: int) -> str:
    """The Pro generators' landscape ratios; a portrait source takes 1:1, the
    only one that does not squash it."""
    if width <= 0 or height <= 0 or height > width:
        return "1:1"
    ratio = width / height
    candidates = {"1:1": 1.0, "4:3": 4 / 3, "3:2": 1.5, "16:9": 16 / 9}
    return min(candidates, key=lambda key: abs(candidates[key] - ratio))

Exercise

Pick a long-running job in your own software that calls a service you do not control, such as an image render, a transcription or a report export. Change it so the job id is written to durable storage the moment the service accepts the job, and so a restarted worker that finds a recorded id resumes waiting instead of submitting again. Treat a 404 while polling as a lost job with its own error, and release or acknowledge the result only after your copy is safely written. Test it by killing the worker between submit and finish and asserting the service saw exactly one submission.

Hint

Look for the recorded id at the very top of the job function, before you build the request. Fake the service with a counter on submit and a list of waited ids, as Spark's test does, so the assertion is two lines.

The last and slowest layer

Spark writes the recipe

Which photo Ember sees

A job that survives a restart

On the screen

Code

External links

Exercise

Progress

Comments 0