Add non-destroying AsyncIterator to Readable streams

Is your feature request related to a problem? Please describe.
The current async-iterator on Readable streams always destroys the stream, when you break or return or an error is thrown. Specifically, this means that if someone wants to iterate over the same stream in different places (for example iterate until the end of part one... then do some things, continue iterating later etc) this is impossible - you have to have exactly one for await...of that you work inside of. In addition, there are other scenarios like Request where iterating over the request ends up closing the response (which might specifically be a bug in the Request iterator, and not really related to destroying the request) and this suggestion would allow bypassing it for other use-cases without adding special specific use-case code (I'm not sure if this is an issue with Duplex streams in general?).

Describe the solution you'd like
I'd like to create another async-iterator (in addition to the existing one) that would give users the option opt in/out of destroying the stream non-error ending of the iteration (any maybe also on stream-error?). The naming could be very explicit, to let users know that they've decided to opt-out of the default behavior, or if an explicit name is not required just added as parameters to the Symbol.asyncIterator method.

Something like this (I haven't thought a lot about the naming):

for await (const chunk of readable.nonDestroyingIterator({ destroyOnError: true, destroyOnReturn: false}) {
}

Looking around createAsyncIterator in readable.js, this wouldn't be too difficult to add, and I would be happy to provide a PR for this.

Describe alternatives you've considered
Building another iterator myself on existing streams.

Definitely 👍 to this, I've also been in situations where I've piped a stream to a dummy PassThrough just so I could iterate over it without destroying the original stream.

In addition, there are other scenarios like Request where iterating over the request ends up closing the response

Actually Request is not destroyed when for await does not break or throw (but for example file stream is) if you meant that:

let http = require("http");

let server = http.createServer(async function(req, res)
{
	for await (let chunk of req)
	{
		await new Promise(r => setTimeout(r, 1000));
	}
	console.log("destroyed", req.destroyed);
	res.end("ok");
});

(async function()
{
	await new Promise(resolve => server.listen(resolve));
	let req = http.request({ port: server.address().port, method: "post" }).end("abc");
	let res = await new Promise((resolve, reject) => req.on("response", resolve).on("error", reject));
	let str = "";
	for await (let chunk of res) str += chunk;
	console.log("client received:", str);
	server.close();
}());

destroyed false
client received: ok

for await (const chunk of readable.iterator({ destroyOnError: true, destroyOnReturn: false }) {
}

Because it's a generic method

stream: add a non-destroying iterator to Readable

Navigation Menu

Repository navigation

Add non-destroying AsyncIterator to Readable streams #38491

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add non-destroying AsyncIterator to Readable streams #38491

Description

Activity

addaleax commented on May 1, 2021

misos1 commented on May 2, 2021

ronag commented on May 2, 2021

ronag commented on May 3, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions