Tree sync

[dapplion]

Currently forwards sync fetches ranges of blocks by range from either head or latest finalized until reaching the target, from a set of peers with the same status messages. This approach has fundamental issues that make the sync complex and susceptible to griefing attacks. (I'll describe those later)

Proposal

Divide forward sync into two phases:

1. Header backfill: construct an in-memory tree from all unknown received by peers (status, unknown attestations, etc) and backfill until reaching a known block or conflicting with finality.
2. Forward sync with knowledge of what block roots to expect at each slot and perfect peer grouping.

This model allows us to address all those issues and delete a lot of code. All lookup sync, by_range routes, batching, optimistic batches, among others, are deleted.

Progress

• [ ] PoC https://github.com/dapplion/lighthouse/pull/46
• [ ] Add headers_by_root p2p spec

Related resources

• Early exploration on modifying the spec to address one of the issues above https://github.com/sigp/lighthouse/pull/7096

[dapplion]

I propose to implement tree-sync progressively instead of in one big rewrite like https://github.com/dapplion/lighthouse/pull/46. The steps I propose next are well scoped and contained, so we will have a much simpler transition introducing the concepts of tree-sync one by one.

Tree sync is basically extending lookup sync to be able to handle very large chains of blocks, replacing the need for range sync. The key changes are:

0. Add more test coverage of sync tests (agnostic of range or lookup sync)
1. When creating a lookup fetch its ancestor headers until one is known or conflicts with finality
2. Introduce a new ReqResp route headers_by_root to make 1. fast and efficient
3. Make lookup sync downloads more efficient by buffering downloads (like range sync does)
4. Make lookup sync processing efficient by processing its components in batch
5. Remove the max-depth limit of lookup and sync, and trigger lookup sync when a peer's status message contains an unknown root
6. Delete range sync