readable-stream icon indicating copy to clipboard operation
readable-stream copied to clipboard

How to implement a through stream

Open msageryd opened this issue 1 year ago • 20 comments

Streams are sometimes hard to grasp for me. The various versions in Node doesn't make it easier.

It seems lika a good idea to use redable-stream so I can at least have a solid ground to build knowledge upon, instead of an ever changing landscape.

I'm reading a stream of binary files (stdout from a child process), the files are delimited with a delimiter. I'm looking at each chunk when it arrives and either just write it to my outputStream or create a new outputStream if a file delimiter is discovered. After that , each new stream needs to be cloned (using cloneable-readable) and piped to different locations.

Originally I had an "outputStreamFactory" which created a writableStream and piped it to it's destination. The streamFactory was used to create a new stream everytime a file delimiter was discovered. This does not work anymore, since I need to pipe the stream again (cannot pipe a writable stream).

Q1: should I use a through stream for this? Q2: readable-stream does not have a through-stream. Should I build one from a transform stream? Q3: in that case, how should I build a through stream safely from a transform stream? Q4: or should I use thgrough2, which is probably not using readable-stream under the hood

msageryd avatar Sep 26 '23 14:09 msageryd