node icon indicating copy to clipboard operation
node copied to clipboard

webstreams: integrate into common stream utils

Open jasnell opened this issue 3 years ago • 15 comments

With the introduction of web streams, it would be good to integrate support into the various common stream utilities...

Refs: https://github.com/nodejs/node/pull/39134

/cc @mcollina @ronag

  • stream.finished()
const { finished } = require('stream');
finished(new ReadableStream(), (err) => { /* ... */ });
finished(new WritableStream(), (err) => { /* ... */ });
finished(new TransformStream(), (err) => { /* ... */ });
import { finished } from 'stream/promises';
await finished(new ReadableStream());
await finished(new WritableStream());
await finished(new TransformStream());
  • stream.pipeline()
const { pipeline } = require('stream');
pipeline(new ReadableStream(), new TransformStream(), new WritableStream(), (err) => { /* ... */ });
import { pipeline } from 'stream/promises';
await pipeline(new ReadableStream(), new TransformStream(), new WritableStream());
  • stream.addAbortSignal()
const { addAbortSignal } = require('stream');
const ac = new AbortController();
const readable = new ReadableStream();
addAbortSignal(ac.signal, readable);
  • stream.compose() (https://github.com/nodejs/node/pull/39029)

jasnell avatar Jul 09 '21 00:07 jasnell

As far as streams goes, this is probably a good first issue.

mcollina avatar Jul 09 '21 06:07 mcollina

Going to take a look at this as my first issue. Is that okay?

ghost avatar Jul 12 '21 08:07 ghost

No need to ask for permission. Just go ahead.

ronag avatar Jul 12 '21 10:07 ronag

Hello, I would like to assist on this issue. Can I get some more documentation and reading

Thokya avatar Jul 26 '21 13:07 Thokya

import { pipeline } from 'stream/promises';
await pipeline(new ReadableStream(), new TransformStream(), new WritableStream());

Adapting pipeline to whatwg:streams feels almost a bit unnecessary, whatwg:streams are already kind of promise based already

await new ReadableStream()
  .pipeThrough(new TransformStream())
  .pipeTo(new WritableStream())

I think i would never use pipeline and whatwg:streams together, I don't know so much about finished either...

jimmywarting avatar Jul 27 '21 17:07 jimmywarting

https://github.com/nodejs/node/pull/39519 did add support for some of the things here but lacked tests so it was disabled. If someone wants to pick up this issue I would recommend looking at https://github.com/nodejs/node/pull/39519, uncomment the web stream stuff, add tests and fix any remaining issue there.

ronag avatar Aug 03 '21 08:08 ronag

Thanks for the heads up. I'll take a look this week :)

ghost avatar Aug 03 '21 09:08 ghost

Adapting pipeline to whatwg:streams feels almost a bit unnecessary, whatwg:streams are already kind of promise based already

The goal should be to add those for cross compatibility, so we can "pipe" from a WHATWG Stream to a Node.js Stream easily and without additional overhead.

mcollina avatar Aug 03 '21 09:08 mcollina

@mcollina does this still need to be handled? I can take it if so, although will take me a while.

evanrittenhouse avatar Mar 21 '22 20:03 evanrittenhouse

is this issue still unresolved ?? Can anyone tell me please??

iSatVeerSingh avatar Aug 18 '22 08:08 iSatVeerSingh

I think this is still unresolved. A PR adding support (or at a minimum tests) for this would be highly appreciated.

The use case these utilities are there to answer is to create interoperability between webstreams and nodestreams.

mcollina avatar Aug 18 '22 09:08 mcollina

@mcollina any documentation I can read to get more information on this? I feel like I can handle this as a first-timer.

Marcellofabrizio avatar Sep 20 '22 13:09 Marcellofabrizio

Not much TBH, however in the simplistic form: start writing a test, and keep working until it passes.

mcollina avatar Sep 20 '22 13:09 mcollina

#39519 did add support for some of the things here but lacked tests so it was disabled. If someone wants to pick up this issue I would recommend looking at #39519, uncomment the web stream stuff, add tests and fix any remaining issue there.

@mcollina maybe pick up after this?

Marcellofabrizio avatar Sep 20 '22 13:09 Marcellofabrizio

yes!

mcollina avatar Sep 20 '22 13:09 mcollina

Can i take up this issue or it is resolved?

vijayabhaskar-ev avatar Oct 18 '22 15:10 vijayabhaskar-ev

go for it

mcollina avatar Oct 18 '22 16:10 mcollina

Implemented in:

  • finished() https://github.com/nodejs/node/commit/32254988ba7c0c0a68d8dfbc32430466370bf170
  • pipeline() https://github.com/nodejs/node/commit/23effb255efe3eb0dc935e3a430d80b41ea1e660
  • addAbortSignal() https://github.com/nodejs/node/commit/96c720e98f4ea80103a9d240ae8072190a226729
  • compose() https://github.com/nodejs/node/commit/94e1f8f8e139680d1de0531d8aaf7f971cfaa953

debadree25 avatar Feb 27 '23 09:02 debadree25