Distributed.jl
Distributed.jl copied to clipboard
memory is not being released aggressively enough during parallel processing
I have a large codebase that runs without any problems in 0.4.0, but shows symptoms of memory leak when running in 0.4.3.
Basically, the memory increases after a call to @sync
@parallel
for. It seems that some of the memory allocated for the parallel processing is not freed. A call to gc() in the end of the loop does not fixes the problem.
The figure below shows the memory allocation for the same script using 0.4.0 (first half of the graphic) and 0.4.3, where it can be seen thtat the memory increases after each call to @sync
@parallel
for.
It is the first time I fill an issue, so if I am not using the proper words/methodology, I apologize.