qcdll
qcdll
it may be worthwhile to keep a multi-day cluster of python3 with #254
here is a slave left overnight: https://gist.github.com/qcdll/fe22b6ab2c143b3bf8f13e8a3f156f6c the log takes snapshot every 1 min and report differences, there is some steady increase from core.py which seems to relate to deserialization...
top mem usage by lineno and some traceback this was obtained from a slave that ran 1+ days https://gist.github.com/qcdll/8a6c1ce4fd5b675162fb6dfb1a0d7fe7
based on analysis above, adding new counts in #254 and will be launching clusters with python3 and #254 for testnet 23 also as @ninjaahhh suggests will also be running profiling...
add the following snippet to `test_cluster.py`, we see what fd are being leaked: a few samples ``` pypy3 63155 dll 159 PIPE 0x60bc88493f1ee7d3 16384 ->0x60bc88493f1f0093 pypy3 63155 dll 160 PIPE...
one of the fixes made was https://github.com/QuarkChain/pyquarkchain/pull/397, but it's a bit hacky and we should audit all places that a file/socket was opened
If I recall correctly, timing out when downloading block header or block will cause the peer sending `new_minor_block_header_list_command` to be disconnected from current cluster, preventing it from performing the same...
should be resolved by #199
in the end, tracemalloc's compare_to() in snapshot helped identify the root cause of memory leak in #252 we can use this PR in the future if memory issue arises again