nethermind icon indicating copy to clipboard operation
nethermind copied to clipboard

Flat / path based state layout

Open damian-orzechowski opened this issue 2 years ago • 0 comments

This is a draft PR with flat / path based implementation of state in NM to show the scope of the changes. This may be split further into different chunks to have more control over the stability.

Changes

  • Implementation of new database for state (named pathState) using 2 columns (RocksDB) to store the state and storage separately
  • Implementation of TrieStoreByPath which is responsible for managing storage and retrieval of trie nodes using path
  • Extending TrieNode to track path of the nodes
  • Implementation of memory cache which holds the changes for the configurable number of last blocks (default 128). The cache is also responsible of tracking the relationship between upcoming blocks to correctly persist and handle re-orgs - PathDataCache
  • Synchronization changes for snap sync and fast sync (for healing). Healing performs range deletion to ensure no dirty data is left

TODO list:

  • refactoring to build a separate IWorldState for path base store using changes from the: #6260, #5533
  • archive storage
  • performance improvements

Types of changes

What types of changes does your code introduce?

  • [ ] Bugfix (a non-breaking change that fixes an issue)
  • [x] New feature (a non-breaking change that adds functionality)
  • [x] Breaking change (a change that causes existing functionality not to work as expected)
  • [x] Optimization
  • [x] Refactoring
  • [ ] Documentation update
  • [ ] Build-related changes
  • [ ] Other: Description

Testing

Requires testing

  • [x] Yes
  • [ ] No

If yes, did you write tests?

  • [x] Yes
  • [ ] No

Notes on testing

Optional. Remove if not applicable.

Documentation

Requires documentation update

  • [x] Yes
  • [ ] No

If yes, link the PR to the docs update or the issue with the details labeled docs. Remove if not applicable.

Requires explanation in Release Notes

  • [ ] Yes
  • [ ] No

If yes, fill in the details here. Remove if not applicable.

Remarks

Optional. Remove if not applicable.

damian-orzechowski avatar Jan 11 '24 08:01 damian-orzechowski