ssz
ssz copied to clipboard
Don't serialize immutable Validator data
Nimbus uses a very nice optimization that we should implement too:
When serializing state, don't serialize + deserialize Validator object fields withdrawal_credentials
and pubkey
. Instead store this data is a separate global append-only list.
- Reduce disk size usage significantly
- Reduce serialization + de-serialization time by +50%
- Ensure data de-duplication when doing regen
To implement in SSZ, Container type must have some option to exclude certain fields from serialization. Then after reading the state, all validator object values must be iterated and mutated to append the immutable properties.
Nimbus references:
- https://github.com/status-im/nimbus-eth2/blob/bef13b6cce244510ba989d3ce2a6176426592604/beacon_chain/beacon_chain_db_immutable.nim#L44
- https://github.com/status-im/nimbus-eth2/blob/bef13b6cce244510ba989d3ce2a6176426592604/beacon_chain/spec/datatypes/base.nim#L282
- https://github.com/status-im/nimbus-eth2/blob/bef13b6cce244510ba989d3ce2a6176426592604/beacon_chain/beacon_chain_db.nim#L514-L524
WIP branch
- https://github.com/ChainSafe/ssz/pull/new/dapplion/container-omit-deserialize