[Fix] Coupling block sync to DAG state
Motivation
This PR focuses on coupling block sync to DAG state replication. When a node is syncing via block responses, it will sync its storage and DAG with the certificates contained in the block and attempt to update its ledger. Previously, there were scenarios where a node would commit certificates in its DAG without advancing blocks. Instead, the committal of leader certificates and advancement of blocks during sync should be tightly coupled.
To achieve this, if a node is syncing, it should forgo committing leader certificates on a rolling basis inside update_dag. Instead, the leader certificate should be committed just before the ledger is ready to advance to the next block during sync. To facilitate this, we use a sender channel that communicates the leader certificate to be committed from the Sync module to the BFT.
A previous version of this PR can be found here https://github.com/AleoNet/snarkOS/pull/3268 .
The differences are that the previous PR does not prevent syncing nodes from committing leader certificates within update_dag, and that certificates inside block responses are added to the DAG all at once when the availability threshold is met. For contrast, in this PR, we maintain the original method of updating the DAG as soon as the certificates in the block response are processed within sync_storage_with_block, but only commit the leader certificate when the ledger is ready to advance to the next block. Updating the DAG as soon as the certificates are processed is necessary to ensure there is no discrepancy between the storage and the DAG state beyond the latest committed block.
To summarize, this PR makes the following changes:
- Implements sender-receiver channels for committing leader certificates and determining recently committed certificates between Sync and BFT.
- Prevents syncing nodes from committing leader certificates within
update_dag. - Instead, commits the leader certificates in the BFT using a sender-receiver channel within
sync_storage_with_blockjust before the ledger advances blocks.
Test Plan
Relevant BFT test cases include the following: 2. BFT-Rebonding 12. Sync-Invalid-Peers-Attack 13. Sync-Far-Behind
Related PRs
https://github.com/AleoNet/snarkOS/pull/3268