Reducing hard drive validator footprint

[ilblackdragon]

Currently validators foot print is very high as they maintain additional indices that are not needed.

There should be a flag to not maintain those indecies and reduce hard drive footprint of validator.

A number of validators OOMed due to over 1.4 TB hard drive during spike in transactions.

[bowenwang1996]

To add more context here, currently a validator node and a rpc node store the same amount of data in rocksdb. However, validator nodes do not need to store some indices such as DBCol::Transactions, DBCol::Receipts, DBCol::TransactionResultForBlock and so on. Removing those indices for validator nodes could help reduce the disk usage of a validator node, which reduces the cost of running a validator node.

[packetstracer]

No brainer decission for us. The less resources needed the better performance and efficiency for the network.

[nagisa]

In order to make an issue a good one for somebody to work on as their first contribution this lacks pointers towards the approximate area of where the changes should be made. If there are any anticipated gotchas, those should be written down too.

[fallenmi]

will check this week

[tayfunelmas]

@valar999 I see you also sent #10748. do you want to assign this issue to yourself if you want to work on it? it is currently unassigned.