flowpipeline
flowpipeline copied to clipboard
feat(main): Add possibility to concurrently spawn pipelines using -j
Just a first draft. Effectively, this spawns j concurrent pipelines that are wired up separately. Hence, it might affect the order of throughput. We could leverage concurrency even better my wiring up the pipeline in a non-static way, i.e. for every segment boundary we have just one channel, with j segments writing and reading to each of these boundary channels.
That'll be more complex, but I'm not sure it is worth the complexity in terms of gain.