fs2 icon indicating copy to clipboard operation
fs2 copied to clipboard

Create another version of groupWithin() which won't create a singleton chunk when elements are available in bulk after being idle for a long time

Open chenjianjx opened this issue 4 months ago • 0 comments

Current groupWithin() has a feature which doesn't fit our requirements.

An example scenario:

  • The stream is started, but no element is available for a long time, longer than the timeout
  • Then a batch of elements are available
  • Since the stream is in "timed out" state, it will emit a chunk right away. It will pick the first element that arrives, and create a singleton chunk.

We need another version of this method . Same scenario:

  • The stream is started, but no element is available for a long time, longer than the timeout
  • Then a batch of elements are available. Let's say the available time of the first element is T0
  • The stream will not emit right away, until the number of elements available reaches chunkSize, or when the time is at T0 + timeout

This is very desirable if the chunk processing logic prefers a big batch, instead of just a few elements.

chenjianjx avatar Sep 12 '25 12:09 chenjianjx