Make the upper bound of memory usage clear

[tkstanczak]

Currently we have an overcomplicated config of caches for each database separately:

1. consider auto-tuning the cache sizes based on 1 - 3 parameters (network, sync mode, abailable physical RAM)

2. Display the expected upper memory bound upon launching the application

3. Include detailed (per db) memory usage metrics in Grafana dashboard

4. Review the impact if column families on the memory settings

[roninkaizen]

[tkstanczak]

the out of memory - it is easily fixable through config and the issue is about making users aware on how to avoid this issue from happening - let me paste some solution here temporarily

[tkstanczak]

If it is goerli.cfg then the best chance to limit this further would be by changing "BlockInfosDbCacheIndexAndFilterBlocks": false to "BlockInfosDbCacheIndexAndFilterBlocks": true

another (more drastic change) would be:

BlockCacheSize": 128000000: to BlockCacheSize": 16000000:

{ "ConfigModule": "DbConfig", "ConfigItems": { "WriteBufferSize": 16000000, "WriteBufferNumber": 4, "BlockCacheSize": 128000000, "CacheIndexAndFilterBlocks": true, "BlockInfosDbCacheIndexAndFilterBlocks": false } },

[roninkaizen]

thanks for the information, will try this and send response, can be closed from my point of view, tested now 12 hours with the provided changes and did not encounter any further notifications like the posted ones

[tkstanczak]

Still outstanding - some work has been started on columns for rocksdb

[kamilchodola]

@asdacap I think this is something I was also referring recently but this one seems to be more detailed. Can you please comment on that?

cc @LukaszRozmej @benaadams

[LukaszRozmej]

I think caches are now explicit, memory hint should be removed and this should be closed.