pyquarkchain icon indicating copy to clipboard operation
pyquarkchain copied to clipboard

memory creeping up

Open ninjaahhh opened this issue 6 years ago • 5 comments

memory is still creeping up on my 4g-mem machine. after running for the 3~4 days (after the keccak fix) the memory usage went from ~1.2g to ~3g.

master: ~700m slave 1: ~1.1g slave 2: ~1.1g

ninjaahhh avatar Dec 10 '18 05:12 ninjaahhh

it may be worthwhile to keep a multi-day cluster of python3 with #254

qcdll avatar Dec 10 '18 05:12 qcdll

agree with @qcdll. let's keep track of those machines

ninjaahhh avatar Dec 10 '18 17:12 ninjaahhh

here is a slave left overnight: https://gist.github.com/qcdll/fe22b6ab2c143b3bf8f13e8a3f156f6c the log takes snapshot every 1 min and report differences, there is some steady increase from core.py which seems to relate to deserialization (mem allocated on those lines were never freed)

qcdll avatar Dec 11 '18 18:12 qcdll

top mem usage by lineno and some traceback this was obtained from a slave that ran 1+ days https://gist.github.com/qcdll/8a6c1ce4fd5b675162fb6dfb1a0d7fe7

qcdll avatar Dec 12 '18 19:12 qcdll

based on analysis above, adding new counts in #254 and will be launching clusters with python3 and #254 for testnet 23

also as @ninjaahhh suggests will also be running profiling with mining enabled

qcdll avatar Dec 13 '18 07:12 qcdll