erigon icon indicating copy to clipboard operation
erigon copied to clipboard

Add snapshot processing for all astrid modes

Open mh0lt opened this issue 1 year ago • 1 comments

This PR allows stand alone astrid to participate in snapshot production.

It includes the following changes:

  • All bor data storage is handled in the bor/heimdall stage
  • Stores now process db & snapshot calls
  • Snapshot production runs from multiple databases in stand alone mode
  • In the legacy stages and astrid stage mode the chain db is still used

The current state of the types which manage bor persistence are as follows:

db-types-Astrid Types

These changes have been tested by running amoy from scratch in various modes:

  • no polygon sync
  • polygon.sync
  • polygon.sync.stage

I have also tested removing chaindata (& heimdall & polygon-bridge) dbs and restarting as well as removing locally produced snapshot files.

In all cases for my testing amoy startes, syncs and runs at the tip.

mh0lt avatar Oct 16 '24 10:10 mh0lt

@mh0lt this is great effort! There are 2 main areas to discuss:

  1. Seems like the pruning flow is missing glue to tie it with the Heimdall and Bridge DBs for when running in standalone mode
    • https://github.com/erigontech/erigon/pull/12339#discussion_r1806733045
    • https://github.com/erigontech/erigon/pull/12339#discussion_r1806736390
  2. Is it possible to have the "Ready" functionality go via ExecutionEngine instead so that we don't have to deal with the BlockReader logic in polygon/sync/execution_client.go?
    • https://github.com/erigontech/erigon/pull/12339#discussion_r1806688809
    • https://github.com/erigontech/erigon/pull/12339#discussion_r1806691865
    • https://github.com/erigontech/erigon/pull/12339#discussion_r1806692919

The other comments are correctness and cleanup related things that ive noticed.

taratorio avatar Oct 18 '24 17:10 taratorio