effect icon indicating copy to clipboard operation
effect copied to clipboard

Accept ReadableStream `queuingStrategy` in `Stream.toReadableStream`

Open mattrossman opened this issue 1 year ago • 0 comments

05-27 edit: better conform to pipe patterns

What is the problem this feature would solve?

When a Stream is converted to a standard ReadableStream with Stream.toReadableStream, the stream immediately begins pulling a value even though no attempts to read the stream have been made. https://effect.website/play#5459e589747e

This is due to the default queuingStrategy of the ReadableStream constructor using ahighWaterMark of 1. https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/ReadableStream#queuingstrategy

This behavior is not always desirable—in my application I need to set a specific eagerness (similar to Stream.buffer), or you may wish for the stream to only pull values when read, matching the behavior of effect's Streams.

What is the feature you are proposing to solve the problem?

I propose making Stream.toReadableStream configurable with a strategy option, which would get passed onto the underlying ReadableStream constructor.

const stream = Stream.make(1,2,3)

// Default queuingStrategy
const readableStream1 = stream.pipe(Stream.toReadableStream)

// Custom queuingStrategy
const strategy = new CountQueuingStrategy({ highWaterMark: 2 })
const readableStream2 = stream.pipe(Stream.toReadableStream({ strategy }))

What alternatives have you considered?

I thought perhaps Stream.toReadableStream could enforce a highWaterMark of 0 and let developers control stream eagerness with Stream.buffer, however that only allows buffering to start after a read has been attempted. Rather, I think ReadableStreams should be able to start pulling immediately but with a configurable limit.

mattrossman avatar May 24 '24 13:05 mattrossman