streamly icon indicating copy to clipboard operation
streamly copied to clipboard

Improve takeByTime

Open harendra-kumar opened this issue 4 years ago • 2 comments
trafficstars

There are two takebyTime combinators, one in IsStream module and the other one in Fold module. Both could be quite inefficient because of the way they are implemented. One use getTime and the other one starts a timer thread for each time window.

We can implement these efficiently by using the more efficient timestamped or other stream timestamping functions like times. We can then use the timestamps to do the time based chunking. Basically, the time based chunking functions should work on an already timestamped stream.

harendra-kumar avatar Dec 06 '20 12:12 harendra-kumar

This will not work as

  1. takeByTime is implemented incorrectly
  2. This combinator should run a supervisor that should terminate the underlying action if time exceeds. I clearly remember talking about this. I remember documenting this somewhere as well but I'm unable to find the doc.

adithyaov avatar Feb 04 '22 01:02 adithyaov

single element stream timeout paallelFst stream

adithyaov avatar Jun 14 '22 14:06 adithyaov

See https://github.com/composewell/streamly/issues/1675

adithyaov avatar Sep 07 '22 06:09 adithyaov