streamly
streamly copied to clipboard
distribute scan combinators with statically generated scan
We have combinators with monadically generated scans which is very powerful but can be inconvenient to use when we just need some static scans. Currently we have:
parDistributeScan :: MonadAsync m => (Config -> Config) -> m [Scanl m a b] -> Stream m a -> Stream m [b]
parDemuxScan :: (MonadAsync m, Ord k) => (Config -> Config) -> (a -> k) -> (k -> m (Scanl m a b)) -> Stream m a -> Stream m [(k, b)]
In the monadic versions if we want one-shot scans we have to use a destructive read so that the scans are deleted once they are used by the consumer. This is inconvenient for simple cases where we just need a static list of scans.
We can change the existing API names to have an M suffix and use these names for simpler non-monadic scan generation function.
Adding more context: Another API suggested was,
parDistributeScan :: MonadAsync m =>
(Config -> Config) -> Stream m [Scanl m a b] -> Stream m a -> Stream m [b]
But I've benchmarked this, and it is comparatively inefficient.