The Media Type Travels With the Bytes

"Bytes without a media type are a sealed box with no label. The adapter needs to know it's a JPEG, not guess."

Bytes Alone Are Ambiguous

A blob of image bytes doesn't announce what it is. The same raw buffer could be a JPEG, a PNG, a WebP, or a HEIC — and each is decoded differently. The media type (the MIME type: image/jpeg, image/png, image/webp, …) is the label that tells the adapter how to interpret the box. Send the bytes without it, or with the wrong one, and you're handing over a sealed package with no shipping label: it might get opened correctly, it might get rejected, it might get misread. "The bytes arrived" is necessary but not sufficient — they have to arrive correctly labeled.

An Inseparable Pair

Treat { bytes, mediaType } as one indivisible value that always travels together, from staging to the adapter. The moment they get separated — bytes carried here, media type inferred there, or defaulted to a guess — you've introduced a class of subtle failures:

Wrong default: assuming everything is image/jpeg quietly corrupts every PNG and HEIC the user actually sent.
Lost on a hop: the media type known at staging but dropped before the adapter forces a downstream guess.
Re-detected inconsistently: two code paths sniffing the type separately can disagree, reviving the drift bug from the shared-resolver lesson.

The fix is to capture the media type at the source, where the file's real type is known, and carry it as an inseparable partner to the bytes all the way through. It's the same discipline as the atomic aggregate, one level down: two pieces that are meaningless apart, kept together by design.

Small Field, Load-Bearing

It's easy to treat the media type as a trivial string and let it fall off somewhere in the pipeline — it's just a few characters. But it's the few characters that turn an anonymous byte buffer into a picture the model can actually decode. This is the last piece of the multimodal promise: not just "real bytes reached the adapter," but "real bytes, correctly labeled, reached the adapter." Get the pairing right and vision is honest end to end; drop the label and you can send perfect bytes that still fail to be seen.

Code

Capture the media type at the source; carry it as a pair·typescript

// The type is known where the file is real. Capture it there.
type ResolvedImage = { bytes: ArrayBuffer; mediaType: string }; // never separated

async function resolveStagedImage(localId: string): Promise<ResolvedImage> {
  const blob = await readStagedBlob(localId);
  return { bytes: await blob.arrayBuffer(), mediaType: blob.type }; // paired here
}

// WRONG: split the pair and re-guess the type downstream.
//   send(bytes);                         // media type left behind
//   const mediaType = "image/jpeg";      // a default that corrupts PNG/HEIC

// RIGHT: the media type rides with the bytes into every envelope, unchanged.

Exercise

Trace an image's media type through a pipeline: file picked, staged, resolved, enveloped, sent to the adapter. At each hop, ask 'is the media type still attached to these bytes?' Find the first place it could get dropped or defaulted. Then fix the design so the media type is captured at the source and travels as an inseparable pair — and describe what breaks for a HEIC photo if it doesn't.

Hint

The media type is most often lost at a hop that only forwards the bytes, forcing a downstream default like 'image/jpeg'. For a HEIC photo, that default means the adapter tries to read HEIC bytes as JPEG and the image fails to decode — perfect bytes, wrong label, no vision.