10 Streams

Streams let JavaScript process data incrementally, chunk by chunk, so applications can start work early and avoid holding entire datasets in memory. The chapter emphasizes that streams are not a single technology but a family of three related approaches: Node.js streams, Web Streams, and async iterables. All three serve the same broad purpose of moving data from producer to consumer, often through transforms, but they differ sharply in how data flow is initiated, how backpressure is handled, and how errors travel through the pipeline.

Node.js streams are built on an event-driven, mostly push-based model, with Readable, Writable, and Transform objects coordinated through buffers, highWaterMark limits, and advisory signals like boolean return values and drain events. Web Streams use a pull-based design centered on promises, controller.desiredSize, and methods such as pipeTo() and pipeThrough(), which provide stronger structural backpressure but impose a per-chunk promise cost. Async iterables are the simplest and strictest model: consumers drive the pace with next() and for await...of, backpressure is implicit because the producer cannot advance until asked, and each step also carries promise overhead plus suspended generator state.

The chapter’s main performance lesson is that the real danger is not choosing the “wrong” stream type, but ignoring the model’s rules. Mixing stream types inside a hot pipeline, extracting a Web Stream controller to push externally, ignoring Node.js backpressure signals, wrapping synchronous work in async transforms, or retaining chunks across iterations can all create hidden allocation pressure, garbage-collection churn, and memory growth. The recommended practice is to stay in one model when possible, let built-in piping mechanisms manage flow and cleanup, match async machinery to genuinely asynchronous work, and remember that for small bounded payloads a plain function may be better than streaming at all.

The fundamental flow of data through a stream is the same regardless of the specific implementation. A producer provides chunks of data that flow through a series of transforms and on to a consumer.

In the push model, the producer controllers the flow of data through the stream, like turning a faucet on to start flowing water through a hose. The consumer reacts to incoming data as quickly as it is able to.

Under the pull model, the data flows only when the consumer specifically asks for it.

The Node.js streams state. A Node.js Readable starts in the null state, where no data is produced. Attaching a data event handler or calling resume() transitions it to flowing, where the stream calls read() repeatedly and emits data events for each chunk. Calling pause() moves it from flowing to paused, where data accumulates in the internal buffer but no data events are emitted. Calling resume() moves it back to flowing. The null state exists because the stream cannot know at creation time whether the consumer will want data pushed automatically or read manually. This three-state design is a source of most confusion around Node.js stream behavior.

Data flow through a Node.js pipeline with a single Transform. Data moves downwards through four internal buffers: the Readable's buffer, the Transform's inbound buffer, the Transform's outbound buffer, and the Writable's buffer. Each buffer has its own highWaterMark. Backpressure propagates upwards: when any buffer exceeds its mark, the upstream stage is signaled to slow down. Every signal is advisory, and every buffer is a point where chunks can accumulate and survive into old generation if the signal is ignored.

The Web Streams pull cycle. The consumer calls read(), which checks the internal queue. If data is available, the read() promise resolves immediately. If the queue is empty or below the highWaterMark, the stream calls pull() on the underlying source. The producer responds by calling controller.enqueue(), which adds data to the queue and resolves the pending read(). The consumer controls the pace: pull() is never called unless the queue has room, and nothing enters the queue unless the consumer has asked for it.

The all-too-common pattern of extracting the controller from the ReadableStream and using it to push data directly into the streams internal buffer independent of calls to pull() can completely break the streams backpressure signaling mechanism.

Promises allocated per 4 KB chunk on Node.js v24. Node.js streams allocate none, paying their per-chunk cost in event dispatch instead; async iterables allocate eight; a Web Stream pipeline allocates twenty-seven. Exact counts vary by runtime, but the ordering holds.

Resident memory for a fast producer feeding a slow consumer on Node.js v24. Honoring backpressure holds memory flat with about one chunk in flight; ignoring it lets the queue grow without bound until memory climbs. The curve shown is a Web Stream with an extracted controller. A Node.js pipeline that ignores the write() return value produces the same runaway. Absolute megabytes vary by machine; the shape does not.

• JavaScript has three streaming architectures that perform the same job with different machinery. Node.js streams are push-based and built on EventEmitter. Web Streams are pull-based and built on promises. Async iterables are pull-based and defined by the language specification. Each sits at a different level of standardization: language, platform standard, and no specification at all.
• Node.js streams signal backpressure through advisory boolean return values from push() and write(). The producer can ignore the signal entirely, and nothing enforces the limit. The internal buffer grows, chunks survive into old generation, and the garbage collector pays for every one of them.
• Web Streams signal backpressure structurally by only calling pull() when the queue has room. This protection breaks when developers extract the controller and enqueue externally, bypassing the pull cycle. The controller's enqueue() method silently accepts data beyond the highWaterMark.
• Async iterables provide backpressure through absence: the producer cannot run until the consumer calls next(). There is no mechanism to bypass this, making backpressure impossible to ignore. The tradeoff is the inability to buffer ahead.
• Every model pays a per-chunk tax. Node.js pays with nextTick deferrals and EventEmitter dispatch. Web Streams pay with promise allocations and microtask scheduling. Async iterables pay with a promise and a result object per iteration. These costs are negligible for I/O-bound work and significant for high-throughput synchronous data.
• Errors propagate differently across the three models: error events in Node.js streams, promise rejections in Web Streams, and thrown exceptions in async iterables. Mixing models in a pipeline means mixing error propagation strategies, and each boundary is a place where errors can be dropped or transformed.
• Conversion between models adds internal buffers, adapter logic, and scheduling overhead. A pipeline that crosses model boundaries multiple times compounds these costs under concurrent load.
• Patterns that undermine stream performance include wrapping synchronous transforms in async functions, retaining references to chunks after processing, and using streams for data that fits comfortably in memory. Working with the streaming model means letting backpressure propagate, letting chunks die after processing, using built-in piping, and picking one model for the entire pipeline.

FAQ