streamly icon indicating copy to clipboard operation
streamly copied to clipboard

Investigate changing the way folds/scans work for concurrency

Open harendra-kumar opened this issue 1 year ago • 0 comments

To make serial and concurrent folds work the same way, "step" function should just change the state and not return the Done/Partial status. The status should be returned independently by the "extract" function. That way for concurrent folds/scans we can just send the inputs in "step" and then gather the results in "extract" asynchronously. With this change we should be able to implement concurrent scan combinators in a modular way, in terms of parEval.

But this may have an impact on fusion characteristics of serial folds.

harendra-kumar avatar Aug 22 '24 00:08 harendra-kumar