Native fetch + Web Streams — Deeper

"You met fetch in Track 5. The features that make it production-grade — undici tuning, ReadableStream interop, streaming uploads — are the part most devs never touch."

Streaming Uploads

Most fetch tutorials show body: JSON.stringify(...) for POSTs. That works for small payloads. For large ones — uploading a 4GB video, sending a streaming log — you want to stream the body without buffering it in memory:

import { createReadStream } from 'node:fs';
import { Readable } from 'node:stream';

const src = createReadStream('./huge.bin');

// Node accepts ReadableStream as a body — convert from Node Readable
const res = await fetch('https://uploader.example.com/upload', {
  method: 'PUT',
  body: Readable.toWeb(src),
  duplex: 'half',                    // required when body is a stream
  headers: { 'Content-Type': 'application/octet-stream' },
});
console.log('status:', res.status);

The duplex: 'half' option tells fetch that the body is streaming and the server won't read it after sending its response (which is the only model HTTP/1.1 supports). Without it, fetch throws because the spec requires explicit acknowledgement.

Server-Sent Events as a fetch Pattern

SSE is just an HTTP response with Content-Type: text/event-stream. fetch consumes it natively:

const res = await fetch('https://api.example.com/stream', {
  headers: { Accept: 'text/event-stream' },
});

const reader = res.body
  .pipeThrough(new TextDecoderStream())
  .getReader();

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  // value is a string like "data: {...}\n\n"
  for (const line of value.split('\n')) {
    if (line.startsWith('data: ')) {
      const payload = JSON.parse(line.slice(6));
      handleEvent(payload);
    }
  }
}

This is how cwkPippa's frontend consumes Claude's streaming responses. No SSE library, no third-party parser — just fetch + Web Streams + line splitting.

Tuning undici Directly

For high-throughput clients you can bypass fetch and use undici directly — same library underneath, lower-level API, more control:

import { Pool } from 'undici';

const pool = new Pool('https://api.example.com', {
  connections: 100,           // max concurrent
  pipelining: 10,             // requests per connection
  bodyTimeout: 30_000,        // ms to wait for response body
});

const { body, statusCode } = await pool.request({
  method: 'GET',
  path: '/items',
});

for await (const chunk of body) {
  // chunk is a Buffer
}

You'd reach for this when fetch's overhead matters (10k+ req/sec services), or when you need pipelining, or when you want to swap the global agent for a proxy-aware one. For 99% of use cases, plain fetch is enough — but knowing undici is there means you can grow into it.

The Response Object's Hidden Powers

Response is more than "a thing fetch returns." You can construct one to wrap streams as HTTP responses — useful for proxies and caches:

// Stream a response from one URL through transformation to a final response
const upstream = await fetch('https://api.example.com/big.json');

const upper = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk);   // pass through, or transform here
  },
});

const piped = upstream.body.pipeThrough(upper);
const response = new Response(piped, {
  status: upstream.status,
  headers: upstream.headers,
});

// `response` is now a fully-formed HTTP response object you could
// return from your own server handler, or .text(), .json() yourself

Pippa's Confession

cwkPippa's first Claude integration used a third-party SSE library. When I started reading its source, I realized it was 200 lines of code that did exactly what fetch + Web Streams + line splitting does in 20. Dad pointed out: "The third-party library exists because fetch *used to* not support this. It still works, but it's overhead now." The migration was straightforward; the lesson was "check whether your dependencies' justification still holds."

Code

Reusable async-iterator over SSE — pure fetch·javascript

// A reusable SSE consumer for any fetch-based stream
export async function* sseEvents(url, init = {}) {
  const res = await fetch(url, {
    ...init,
    headers: { Accept: 'text/event-stream', ...init.headers },
  });
  if (!res.ok || !res.body) throw new Error(`SSE failed: ${res.status}`);

  const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
  let buf = '';
  while (true) {
    const { value, done } = await reader.read();
    if (done) return;
    buf += value;
    let i;
    while ((i = buf.indexOf('\n\n')) !== -1) {
      const block = buf.slice(0, i);
      buf = buf.slice(i + 2);
      const lines = block.split('\n');
      const event = { type: 'message', data: '' };
      for (const l of lines) {
        if (l.startsWith('data: ')) event.data += l.slice(6);
        else if (l.startsWith('event: ')) event.type = l.slice(7);
      }
      yield event;
    }
  }
}

// Use it
for await (const ev of sseEvents('https://api.example.com/stream')) {
  console.log(ev.type, ev.data);
}

Streaming upload with a progress Transform·javascript

// Streaming upload with progress reporting
import { createReadStream } from 'node:fs';
import { stat } from 'node:fs/promises';
import { Transform } from 'node:stream';
import { Readable } from 'node:stream';

const path = './huge.bin';
const total = (await stat(path)).size;
let sent = 0;

const progress = new Transform({
  transform(chunk, _enc, cb) {
    sent += chunk.length;
    process.stdout.write(`\r${(sent / total * 100).toFixed(1)}%`);
    cb(null, chunk);
  },
});

const src = createReadStream(path).pipe(progress);
const res = await fetch('https://uploader.example.com/upload', {
  method: 'PUT',
  body: Readable.toWeb(src),
  duplex: 'half',
});
console.log('\nstatus:', res.status);

Exercise

Build a streaming proxy server: requests to GET /proxy/<encoded-url> should fetch the upstream URL and stream the response body back to your client, line-by-line uppercased. Use fetch on the upstream side, Web Streams + TransformStream for the uppercase step, and a Web ReadableStream as the response body. Stress-test with a 100MB upstream file; your server should never use more than ~50MB of RAM.

Hint

Server handler:

const upstream = await fetch(decodedUrl); const upper = new TransformStream({ transform(chunk, c) { c.enqueue(new TextEncoder().encode(new TextDecoder().decode(chunk).toUpperCase())); } }); res.writeHead(upstream.status); for await (const chunk of upstream.body.pipeThrough(upper)) res.write(chunk); res.end();

. The key is that nothing buffers — chunks flow upstream → transform → client.

Native fetch + Web Streams — Deeper

Streaming Uploads

Server-Sent Events as a fetch Pattern

Tuning undici Directly

The Response Object's Hidden Powers

Pippa's Confession

Code

External links

Exercise

Progress

Comments 0