streamly icon indicating copy to clipboard operation
streamly copied to clipboard

Implement Array Stream builder

Open harendra-kumar opened this issue 5 years ago • 2 comments

See #701, #700, #660. Array stream builder is the next level after an array builder. We can build a stream of arrays or write an array as soon as it is filled, yield it and start filling another array.

harendra-kumar avatar Sep 24 '20 20:09 harendra-kumar

We can extend Builder t a to Builder t m a for array write effects. If a is Storable the builder instance can write to an Array and later return a stream from the Array. That way we can transparently back the stream by an Array, using the same unified interface but providing unboxed array like performance. We can either yield a stream of arrays or a stream of elements. We may need to control the array chunk size via a state.

harendra-kumar avatar Sep 25 '20 14:09 harendra-kumar

We have a buildChunks fold to build a stream of arrays. However, as suggested in #701 for building an array, similarly we can have a builder monad to build a stream of arrays.

harendra-kumar avatar May 15 '25 09:05 harendra-kumar