flowpipeline icon indicating copy to clipboard operation
flowpipeline copied to clipboard

feat(main): Add possibility to concurrently spawn pipelines using -j

Open debugloop opened this issue 1 year ago • 0 comments

Just a first draft. Effectively, this spawns j concurrent pipelines that are wired up separately. Hence, it might affect the order of throughput. We could leverage concurrency even better my wiring up the pipeline in a non-static way, i.e. for every segment boundary we have just one channel, with j segments writing and reading to each of these boundary channels.

That'll be more complex, but I'm not sure it is worth the complexity in terms of gain.

debugloop avatar Dec 07 '24 09:12 debugloop