vocdoni-node icon indicating copy to clipboard operation
vocdoni-node copied to clipboard

bug: Memory leak in arbo-based vocdoni-node

Open ed255 opened this issue 3 years ago • 1 comments

There seems to be a memory leak in the node that has appeared after integrating the arbo-based StateDB and arbo-based census manager. This affects gateways and miners. As seen in the dev environment, over time, as processes are created and votes made, the memory consumption increases and doesn't drop.

ed255 avatar Oct 01 '21 09:10 ed255

After a heap profile analysis, we observed 3 main sources of memory consumption:

  1. badgerdb.Txn. There's a big memory usage related to WriteTxn resources that should be released after the Txn is used. This could mean that there are some Txns that are not Committed nor Discarded, leading to a memory leak.
  2. Tendermint Block Unmarshal: This one comes from the Block cache which currently stores the last 128 blocks. If we have blocks with many transactions, for some time this cache will take a lot of memory. A possible solution would be to reduce the cache size, or to adapt the cache size to the size of each block, so that the cache takes a constant amount of memory. This doesn't seem to be a leak.
  3. baddgerdb Arena. BadgerDB just uses an Arena as an optimization, but it doesn't grow so it doesn't seem to be a leak.

ed255 avatar Oct 01 '21 16:10 ed255

i haven't seen this happen again, closing

altergui avatar Mar 06 '24 15:03 altergui