reactor
Reactor core primitives Flux#using, Flux#usingWhen, Flux#fromIterable should be GC friendly to prevent memory leak
Expected Behavior
I have quite complicated flux "pipe" that uses "large" (~8MB) memory buffers and Iterables which references those buffers. To be more specific I reads avro files from memory using https://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileStream.html and adapts it to Flux with Flux#fromIterable
I wrapped creation of DataFileStream in Flux#using which then just calls Flux#fromIterable.
I'm expecting resources created with Flux#using to be ready for Garbage Collection right after resourceCleanup is called.
Talking about reactor:
Flux#using and Flux#usingWhen subscriptions should not hold reference to resource and corresponding publisher after resource is cleaned up.
Flux#fromIterable subscriptions should not hold reference to iterator after it is exhausted
Actual Behavior
Mentioned primitives holds strong references to resources/iterators preventing them from being garbage collected. If something holds references to subscriptions then resources may not be garbage collected even if they are not required anymore. To overcome this issue I have to create some strange wrappers which wraps Resources, Iterable and Iterators.
Steps to Reproduce
The minimal reproduceable example I can create: https://gist.github.com/alopukhov/50a8d2e953346692e2c4c819c7c7d065 Please limit heap to 128 MB. It doesn't produce OOM only with specific resource wrapper (testWrapper2).
Possible Solution
Nullify references to resources after they are not required anymore.
Your Environment
- Reactor version(s) used: reactor-core 3.4.23, 3.4.7
Hey, thanks for the report. It has not been noticed for quite a while, so I do apologize on behalf of the team. I will keep it in the backlog since it looks like a valid concern, but we'd appreciate some help if anyone is interested in contributing.
Hey @chemicL , I'd like to help this one. I've implemented the changes and can open a PR for review.
