Distributed.jl icon indicating copy to clipboard operation
Distributed.jl copied to clipboard

memory is not being released aggressively enough during parallel processing

Open CodeLenz opened this issue 8 years ago • 32 comments

I have a large codebase that runs without any problems in 0.4.0, but shows symptoms of memory leak when running in 0.4.3.

Basically, the memory increases after a call to @sync @parallel for. It seems that some of the memory allocated for the parallel processing is not freed. A call to gc() in the end of the loop does not fixes the problem.

The figure below shows the memory allocation for the same script using 0.4.0 (first half of the graphic) and 0.4.3, where it can be seen thtat the memory increases after each call to @sync @parallel for.

processos

It is the first time I fill an issue, so if I am not using the proper words/methodology, I apologize.

CodeLenz avatar Mar 11 '16 23:03 CodeLenz