website icon indicating copy to clipboard operation
website copied to clipboard

Feedback for “Operations” (docs/400-guides/660-streaming/100-stream/400-operations.mdx)

Open florianbepunkt opened this issue 1 year ago • 1 comments

Hi, what I miss from the stream docs is the "other" side of interoperability. How can I get from an effect Stream back to a "regular" NodeJS stream. We have a lot of scenarios, where we do something like

ReadableStream (API, CSV file, etc) ---> Data Processing ---> Writable Stream (S3 Upload).

Effect is great for the data processing step, but for large data sources, I need to stream the data back (s3 upload in the workflow above), but I have no idea how to get from effect's data stream back to a node stream.

florianbepunkt avatar Aug 03 '24 13:08 florianbepunkt

There is a module in /platform-node for working with node streams https://effect-ts.github.io/effect/platform-node-shared/NodeStream.ts.html

tim-smart avatar Aug 03 '24 21:08 tim-smart