realtps icon indicating copy to clipboard operation
realtps copied to clipboard

Memory leak in import job?

Open brson opened this issue 3 years ago • 6 comments

There's probably a memory leak in the import job. Recently the entire machine came down after the import process was killed, and just now I see it is using 30% of the machine's memory. It should be using just a low constant amount of memory.

brson avatar Sep 12 '22 18:09 brson

I just don't see anything in the code that could leak. There's no caching, no collections, tasks are all joined. Maybe one of the chain clients has a leak.

Probably need to use a heap profiler.

Other jobs use low memory.

brson avatar Sep 12 '22 19:09 brson

I know people have used massif for heap profiling rust.

brson avatar Sep 12 '22 19:09 brson

Import job was taking 50% of ram when I logged in.

brson avatar Sep 28 '22 15:09 brson

Memory usage looks much better after some dependency upgrades.

brson avatar Oct 15 '22 20:10 brson

Memory usage for the import job was at a steady low number, but today I noticed that Arbitrum fell behind, and memory usage is at 27%.

Is there some memory being held by import jobs that run forever? I don't know what it would be.

brson avatar Oct 27 '22 21:10 brson

Memory usage immediately rises to 20% again.

brson avatar Oct 27 '22 23:10 brson