Thoughts on memory issues

[rayn316]

The memory usage of erigon when it is provided as RPC has always been very high. Regardless of whether the server is 64G or above 200G, the memory will be full within a few days and cause OOM. When I used pprof to obtain memory a long time ago, I found that the filter uses a lot of memory, but sometimes it is not the filter that uses the most memory. After trying to add new detections to my software, which will automatically delete filters that have not been used for a long time, the memory usage is under stable control. Although erigon still takes up a lot of 60-80G of memory, it is already very good in terms of stability. So I think the team can optimize the memory in this direction, such as optimizing the memory usage of filters, because in geth, filters have little impact on memory.

Filter: json-rpc eth_newFilter eth_newPendingTransactionFilter ...

[mbwmbw1337]

Watching this... I have also had OOM issues luckily have been able to create large enough swap to work around it.

[luarx]

It would be great if memory consumption could be reduced 🙏