CERMINE Apparent memory leak when running multiple workers

Apparent memory leak when running multiple workers

Open rueberger opened this issue 7 years ago • 3 comments

I'm experiencing a memory leak when running cermine with multiple workers, on a large set of PDFs.

java -cp cermine-impl-1.13-Ian2P-jar-with-dependencies.jar pl.edu.icm.cermine.ContentExtractor -path /data/path -outputs jats,text,zones -workers 4

Jan 18 '18 05:01 rueberger

OK now I'm not sure if it's actually a leak... closing for more experimentation...

Jan 18 '18 06:01 rueberger

Upon further investigation, there is definitely a memory leak.

Initial memory use is around 5G, and just creeping up overnight.

Jan 19 '18 00:01 rueberger

Hi @rueberger , could you specify the exact version you are using? 1.13 doesn't seem to have option "-workers".

Mar 09 '18 21:03 dtkaczyk