async-stream
async-stream copied to clipboard
what about when you *need* a parallel operation
say, a merge stream, that takes N sorted streams and merges them into one stream? (it would pull from each stream, but output chunks in correct order - for example, to do a merge sort, or LSMT compaction)
A simpler one would just to merge multiple streams into one, but not control the order.
I suppose that should need to be written. Take a look at co-cat which is useful to concatenate strings and could be used as a guide to write a module that does what you are asking for.
co supports yielding arrays for executing in parallel, so something like this should work:
var chunks = yield [
streamA(end),
streamB(end),
streamC(end)
]
I didn't know that! hmm, I suspect you need to use straight callbacks in this case, because what if streamA calls back 3x as fast as streamB which calls back 3x as fast as streamC. you want to get 9 out of streamA for each item from streamC.
Aaah didn't read you wanted to get multiple chunks per stream.
Something like this:?
var cat = require('co-cat');
var chunks = yield cat([
read(n, streamA),
read(n, streamB),
read(n, streamC)
]);
function read(n, fn){
var todo = +n;
return function*(end){
var data = yield fn(end);
if (!end && data && todo--) return data;
}
}