streamly
streamly copied to clipboard
composewell
Add a `done` action in the Fold type
#531 added exception handling to folds, however it has an issue. The after, finally and bracket functions run the release action on extract. But extract can be called over and over again during a scan. Its not right if we release a resource during extract.
One possible improvement to this would be when we have terminating folds, we can run the release action on Stop. However when the stream ends there is no way to communicate to the fold to do any cleanup. To be able to do that we need a done action in the fold which will always be called by the fold driver when the stream ends. This can help the fold perform any final cleanup.
Also, see the scanlMAfter' combinator introduced by #542 . It also introduces a similar done action for the scan.
If the cleanup is removed from extract, will the garbage collector not do the cleanup anyway (due to the “finalized IORef”)?
(Either the only purpose with the proposed done is to do the cleanup even earlier than the GC would do it, or I’m misunderstanding something.)
This is not only about resource allocation and release.
The fold may have some state (buffering) in the accumulator which it wants to flush when the stream ends. Consider this example - say, we have a fold that groups the key value pairs int he input stream into "sessions" based on the key. The sessions are ejected on a timeout. However, when the stream ends we want to flush all pending sessions immediately.
To tackle this case we (the fold driver) need to tell the fold that now we are done you can do your final processing and give us everything you have. For such a case we need done.
Thanks!
In fact, I shortly afterwards came across a similar need for this feature myself. (Committing a database transaction, as opposed to aborting it, when a fold stops.)
I eagerly yet patiently await this proposed feature. 🙂
If we add such an action to folds we should also consider the same for streams for duality. It may be less intrusive to implement this if we use special types with support for such an action. The types can be inter-converted to the standard types. If we find it worth introducing in the standard types we can do that later.
