Native fetch + Web Streams — Deeper · Node.js Quest

"Track 5 introduced fetch. Production work starts where the short examples stop: streaming request bodies, chunk boundaries, Web Stream interop, and knowing when the lower-level undici package is worth adding."

Streaming Uploads

body: JSON.stringify(...) is fine for a small POST. A large video or a continuous log should move through memory in chunks instead of being buffered in full.

import { createReadStream } from 'node:fs';
import { Readable } from 'node:stream';

const src = createReadStream('./huge.bin');

const res = await fetch('https://uploader.example.com/upload', {
  method: 'PUT',
  body: Readable.toWeb(src),
  duplex: 'half',
  headers: { 'Content-Type': 'application/octet-stream' },
});
console.log('status:', res.status);

Node requires duplex: 'half' when a fetch request body is a stream. The option acknowledges the streaming request shape; omitting it makes Node throw before sending. It does not mean the whole file is buffered first—the ReadableStream still supplies chunks as the request is written.

SSE Is a Streaming HTTP Response

Server-Sent Events uses an HTTP response with Content-Type: text/event-stream. fetch exposes the response body as a Web Stream, but network chunks do not promise to end on line boundaries. Keep the unfinished tail between reads.

const res = await fetch('https://api.example.com/stream', {
  headers: { Accept: 'text/event-stream' },
});
if (!res.ok || !res.body) throw new Error(`SSE failed: ${res.status}`);

const reader = res.body
  .pipeThrough(new TextDecoderStream())
  .getReader();

let pending = '';
while (true) {
  const { value, done } = await reader.read();
  if (done) break;

  pending += value;
  const lines = pending.split('\n');
  pending = lines.pop() ?? '';

  for (const line of lines) {
    if (line.startsWith('data: ')) {
      handleEvent(JSON.parse(line.slice(6)));
    }
  }
}

This minimal pattern is enough for a stream that sends one JSON value per data: line, and it correctly carries a line split across chunks. Full SSE also supports multi-line data, event names, IDs, retries, comments, and CRLF framing; use a complete parser when the server relies on those parts of the protocol.

Use `undici` Directly When You Need Its Controls

Node's built-in fetch is powered by undici, but importing Pool from undici means adding the package as a direct dependency. That trade can be worthwhile when a client needs explicit connection pools, pipelining, timeouts, or dispatcher control.

import { Pool } from 'undici';

const pool = new Pool('https://api.example.com', {
  connections: 100,
  pipelining: 10,
  bodyTimeout: 30_000,
});

const { body, statusCode } = await pool.request({
  method: 'GET',
  path: '/items',
});

for await (const chunk of body) {
  // chunk is a Buffer
}

Start with built-in fetch. Add undici only after the application needs a control that the fetch surface does not expose, then benchmark the real workload instead of assuming the lower-level API is automatically faster.

`Response` Can Wrap a Stream

Response is not only a value returned by fetch. You can construct one around a transformed stream, which is useful in proxy, cache, and server-handler code.

const upstream = await fetch('https://api.example.com/big.json');

const upper = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk);
  },
});

const piped = upstream.body.pipeThrough(upper);
const response = new Response(piped, {
  status: upstream.status,
  headers: upstream.headers,
});

// `response` can be returned by a compatible server handler,
// or consumed with .text(), .json(), and the other Body methods.

Pippa's Confession

cwkPippa's first Claude integration used a third-party SSE library. Reading its source showed me that the core path was fetch, Web Streams, and framing logic. Dad's question was the useful one: "Which protocol cases does the dependency handle that our stream actually uses?" For a deliberately simple one-line JSON stream, a small buffered reader can be enough. For the complete SSE protocol, keeping a tested parser is cheaper than quietly reimplementing every edge case.

Code

Reusable async-iterator over SSE — pure fetch·javascript

// A reusable SSE consumer for any fetch-based stream
export async function* sseEvents(url, init = {}) {
  const res = await fetch(url, {
    ...init,
    headers: { Accept: 'text/event-stream', ...init.headers },
  });
  if (!res.ok || !res.body) throw new Error(`SSE failed: ${res.status}`);

  const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
  let buf = '';
  while (true) {
    const { value, done } = await reader.read();
    if (done) return;
    buf += value;
    let i;
    while ((i = buf.indexOf('\n\n')) !== -1) {
      const block = buf.slice(0, i);
      buf = buf.slice(i + 2);
      const lines = block.split('\n');
      const event = { type: 'message', data: '' };
      for (const l of lines) {
        if (l.startsWith('data: ')) event.data += l.slice(6);
        else if (l.startsWith('event: ')) event.type = l.slice(7);
      }
      yield event;
    }
  }
}

// Use it
for await (const ev of sseEvents('https://api.example.com/stream')) {
  console.log(ev.type, ev.data);
}

Streaming upload with a progress Transform·javascript

// Streaming upload with progress reporting
import { createReadStream } from 'node:fs';
import { stat } from 'node:fs/promises';
import { Transform } from 'node:stream';
import { Readable } from 'node:stream';

const path = './huge.bin';
const total = (await stat(path)).size;
let sent = 0;

const progress = new Transform({
  transform(chunk, _enc, cb) {
    sent += chunk.length;
    process.stdout.write(`\r${(sent / total * 100).toFixed(1)}%`);
    cb(null, chunk);
  },
});

const src = createReadStream(path).pipe(progress);
const res = await fetch('https://uploader.example.com/upload', {
  method: 'PUT',
  body: Readable.toWeb(src),
  duplex: 'half',
});
console.log('\nstatus:', res.status);

Exercise

Build a streaming proxy server: requests to GET /proxy/<encoded-url> should fetch the upstream URL and stream the response body back to your client, line-by-line uppercased. Use fetch on the upstream side, Web Streams + TransformStream for the uppercase step, and a Web ReadableStream as the response body. Stress-test with a 100MB upstream file; your server should never use more than ~50MB of RAM.

Hint

Server handler:

const upstream = await fetch(decodedUrl); const upper = new TransformStream({ transform(chunk, c) { c.enqueue(new TextEncoder().encode(new TextDecoder().decode(chunk).toUpperCase())); } }); res.writeHead(upstream.status); for await (const chunk of upstream.body.pipeThrough(upper)) res.write(chunk); res.end();

. The key is that nothing buffers — chunks flow upstream → transform → client.

Native fetch + Web Streams — Deeper

Streaming Uploads

SSE Is a Streaming HTTP Response

Use `undici` Directly When You Need Its Controls

`Response` Can Wrap a Stream

Pippa's Confession

Code

External links

Exercise

Progress

Comments 0

Streaming Uploads

SSE Is a Streaming HTTP Response

Use undici Directly When You Need Its Controls

Response Can Wrap a Stream

Pippa's Confession

Code

External links

Exercise

Progress

Use `undici` Directly When You Need Its Controls

`Response` Can Wrap a Stream