C.W.K.
Stream
Lesson 05 of 05 · published

Web Streams in Node + Async Iteration

~12 min · streams, web-streams, async-iter

Level 0Node Curious
0 XP0/40 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"Node has two stream ecosystems coexisting in 2026. Knowing how to convert between them is mostly what the runtime gives you for free."

Why Web Streams Are Here

Node Streams predate Web Streams by a decade. When the W3C / WHATWG specified Web Streams for browsers, Node had two choices: ignore them (and become incompatible) or implement them (and have two stream systems). Node chose to implement. The native fetch() in Node, response bodies, file-system blob.stream(), and the modern Web Crypto APIs all use Web Streams. Node's own ecosystem still leans on Node Streams.

The interfaces:

  • ReadableStream — analogous to Node's Readable
  • WritableStream — analogous to Writable
  • TransformStream — analogous to Transform

Different method names, similar concepts. The good news: Node provides converters and both implement [Symbol.asyncIterator].

The Async Iterator Bridge

Every Web ReadableStream in Node 20+ supports for await...of directly:

const res = await fetch('https://api.example.com/large.json');

// res.body is a Web ReadableStream
for await (const chunk of res.body) {
  console.log('got', chunk.byteLength, 'bytes');
}

Same syntax as a Node Readable. The runtime handles the protocol differences. This is the most common path you'll touch — consuming fetch responses chunk by chunk.

Converting Between Worlds

Node provides explicit converters:
  • Readable.toWeb(nodeReadable)ReadableStream
  • Readable.fromWeb(webReadable) → Node Readable
  • Writable.toWeb(nodeWritable)WritableStream
  • Writable.fromWeb(webWritable) → Node Writable
You convert when you need an API from one world that the other doesn't have. Example: you want to gzip a fetch response body. zlib.createGzip() is a Node Transform, but response.body is a Web ReadableStream. Convert one of them.

Pipelines Across Both Worlds

stream.pipeline in Node 22+ accepts Web Streams as stages. Mixing the two is no longer an explicit convert step:

import { pipeline } from 'node:stream/promises';
import { createWriteStream } from 'node:fs';
import { createGzip } from 'node:zlib';

const res = await fetch('https://api.example.com/large.json');

// res.body (Web), createGzip() (Node), createWriteStream() (Node)
// pipeline accepts all of them in one chain
await pipeline(
  res.body,
  createGzip(),
  createWriteStream('cached.json.gz')
);

The runtime quietly wraps the Web ReadableStream with the right adapter. You just pipe things.

When to Reach for Which

Default to Node Streams in Node code — the entire ecosystem (fs, http, zlib, crypto, child_process) speaks Node Streams natively. Reach for Web Streams when:

  • The API you're calling returns one (fetch responses, blob.stream, modern Web Crypto).
  • You're writing code that needs to be cross-runtime portable (Cloudflare Workers, Deno, Bun all speak Web Streams natively).
  • You're building APIs the browser will also consume.

TransformStream — Web's Answer to Transform

Web's TransformStream is cleaner to construct than Node's Transform:

const upper = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk.toString('utf-8').toUpperCase());
  },
});

// Use it in a Web-streams pipeline
await someReadableStream
  .pipeThrough(upper)
  .pipeTo(someWritableStream);

No cb parameter, no this.push; you call controller.enqueue for output. The API is more uniform with pipeTo/pipeThrough instead of Node's .pipe(). If you write code that needs to also run in browsers or Workers, default to TransformStream.

Pippa's Confession

For a long time I treated Web Streams as "the browser version, irrelevant in Node." Wrong. The moment I started writing code that called fetch and processed the response body, I was in Web Streams territory whether I noticed or not. Dad pointed out: "The runtime's job is to make this work. Your job is to know which surface you're on so you can read the docs of the right API." Now I check: am I getting a ReadableStream or a Readable? The docs are different; the methods are different; the converters exist; use them.

Code

Convert + consume — Web ReadableStream → JSONL events·javascript
// Streaming JSONL parse from a fetch response — Web Streams in, Node Transform via convert
import { Readable } from 'node:stream';

const res = await fetch('https://api.example.com/events.jsonl');
if (!res.body) throw new Error('no body');

// Convert Web ReadableStream → Node Readable for compatibility
const nodeStream = Readable.fromWeb(res.body);

// Build a quick JSONL parser as an async generator
async function* parseJsonl(byteStream) {
  const decoder = new TextDecoder();
  let buf = '';
  for await (const chunk of byteStream) {
    buf += decoder.decode(chunk, { stream: true });
    const lines = buf.split('\n');
    buf = lines.pop();   // keep last partial line
    for (const line of lines) {
      if (line) yield JSON.parse(line);
    }
  }
  if (buf) yield JSON.parse(buf);
}

for await (const event of parseJsonl(nodeStream)) {
  console.log(event);   // each JSONL row as a parsed object
}
Pure Web Streams — transform + re-serve·javascript
// Pure Web Streams — a TransformStream pipeline
const res = await fetch('https://api.example.com/big.txt');

const upper = new TransformStream({
  transform(chunk, controller) {
    // chunk is a Uint8Array; decode, transform, re-encode
    const text = new TextDecoder().decode(chunk).toUpperCase();
    controller.enqueue(new TextEncoder().encode(text));
  },
});

// pipeTo a Web WritableStream — here we sink into a Response for downstream
const transformed = res.body.pipeThrough(upper);

// Use it like a real Response (works for re-serving, caching, etc.)
const out = new Response(transformed, { headers: res.headers });
console.log(await out.text());

External links

Exercise

Fetch a large remote JSON file (any public API that returns lots of records is fine — a paginated GitHub events feed works). Pipe its body through gzip and write to disk, all without ever buffering the entire response. Try TWO implementations: (1) convert res.body to a Node Readable and use pipeline with createGzip and createWriteStream; (2) use a Web TransformStream for the gzip-equivalent step and pipeTo a Web WritableStream wrapping a Node Writable. Compare LOC, readability, and which dependencies (zlib vs CompressionStream).
Hint
For (1): await pipeline(Readable.fromWeb(res.body), createGzip(), createWriteStream('out.gz')). For (2): use new CompressionStream('gzip') (Web standard, Node 22+) and res.body.pipeThrough(new CompressionStream('gzip')).pipeTo(Writable.toWeb(createWriteStream('out.gz'))). Both work; option (2) is portable to other runtimes.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue
💛 by Ttoriwarm

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.