harmony icon indicating copy to clipboard operation
harmony copied to clipboard

Optimize beacon chain sync by only syncing the epoch blocks.

Open rlan35 opened this issue 3 years ago • 0 comments

Problem/limitation At Hand

Beacon chain or shard 0 is the chain every nodes in the shard 1 2 3 needs to synchronize with in order to know the latest sharding structure or committee configuration. Currently, nodes in shard 1 2 3 are simply utilizing the existing blockchain sync mechanism to fully download and verify all the blocks in beacon chain, which is very costly both for computation and storage (shard 0 is taking hundreds of GB of disk).

Validators in shard 1 2 3 are complaining that they are storing too much data as they need to store both their own chain in shard 1 2 3 and the beacon chain. Besides, beacon chain is broadcasting every new block to the whole network so nodes can catch up to the latest blocks real time, which also causes lots of p2p network traffic.

In fact, only the epoch block of beacon chain is needed for the nodes in other shards to know about the epoch transition and committee configuration, because this information only changes at the epoch block (between epochs).

Proposed Solution

Instead of synchronizing with every single block in beacon chain, only download and synchronize the epoch blocks for beacon chain (which happens every 18.2 hours/epoch).

  1. Change the beacon block broadcasting logic to only broadcast epoch blocks.
  2. For nodes in shard 1 2 3, only listen to epoch blocks and verify the epoch block based on the signature on the block. No need to do the formal state transition check here since we are not verifying all the blocks. Once the block is verified, read the "shard state" or the committee configuration for the new epoch and do epoch transition just as before using the new shard state.
  3. Update the chain sync logic to be able to synchronize the beacon chain epoch blocks from genesis. Basically download all the epoch blocks from genesis and then verify them in sequence until it hits the latest epoch block in beacon chain.

rlan35 avatar Feb 09 '22 20:02 rlan35