espresso-sequencer
espresso-sequencer copied to clipboard
Fix fee catchup after restart
This PR reproduces and will fix an issue discovered in staging, in which, after a staggered restart of all nodes, the network has forgotten the fee state for all accounts except those which actively produced blocks during the restart. Since the state is needed for undecided blocks, it cannot directly be obtained from persisted merklized state, which is only available for decided blocks.
This PR:
- Adds a restart-style test reproducing the issue
- Adds a new catchup endpoint to fetch multiple accounts at once, for performance
- Modifies SQL-based fee account catchup to reconstruct fee state by searching back to the last decided state and replaying headers
- Caches results of successful reconstruction in HotShot's validated state map
- Decouples storage from API layer by creating
CatchupStoragetrait, distinct fromCatchupDataSource. This allows us to get more information when querying storage, such as the leaf corresponding to fetched account state, which in turn ensures we are always able to cache fetched state in the HotShot view map - Adds a background task to fetch missing quorum proposals that are parents of proposals we have. This ensures that every node will eventually have every proposal in a chain between valid proposals and the anchor, which is required for replay to work
- UPDATES HOTSHOT TO A NON-RELEASE TAG -- let's figure out what to do about that before merging