[FALLBACK] - Only Get Information As Needed on Libp2p when it is a Fallback

[bfish713]

What is this task and why do we need to work on it?

Detailed plan on Notion: https://www.notion.so/espressosys/Libp2p-Fallback-cea28d2cf806466da0699a658ed312eb

Summary

When libp2p is a fallback network it will currently consume a ton of bandwidth. Each round a leader will gossip a DAC, Proposal, and all VID shares to all participants. Proposal and DAC is lightweight so probably not a problem. VID shares could be/probably is a massive throughput bottleneck. Also with https://github.com/EspressoSystems/HotShot/issues/1696 we want to only send each node their specific share. This cannot really be done efficiently with gossip, and direct messaging even 100 nodes in parallel would likely also bottleneck the leader.

Further the leader will have to send the DA proposal directly to the committee because dynamic subscribe/unsubscribe from gossip topics is likely too slow to for non DA committee members to join and broadcast fast enough in a view. We could also be smart about when to join the topic but still this is more overhead on every view which is wasted on the happy path and will not work with rotating committees.

Finally for voting, every node must open a connection to the leader to send their vote. For 100 nodes this is a lot but manageable, with much larger networks this will become an issue. There are a few ways we could try to solve this as well but it's out of scope for this task.

The goal of this task is to have non leader nodes only get what they actually need over the fallback. For censorship resistance and resillience to CDN failures I think what we need is (only considering honest nodes, others can act however they want):

• All QuorumProposals are gossiped by the leader optimistically
• DACs can also be gossiped optimistically on the fallback
• Votes can be directly sent back to the leader but the node can make a choice to try or not.
• Nodes can ask other nodes for the required info to vote
  • If they are DA nodes and get a proposal they can request the DA Proposal from the leader (VID shares can be calculated from the proposal)
  • If non DA nodes, they can request VID shares from the Leader or any peer who claims to serve it.
  • Peers who get the full shares from the CDN can advertise via the DHT that they can serve this info
• All viewsync messages should be optimistically sent via broadcast for certificates, and direct message to the relay for votes
  • If view sync is entered the assumption is the CDN might be slow/down and throughput of View sync is not critical

What work will need to be done to complete this task?

• [x] https://github.com/EspressoSystems/HotShot/issues/2645
• [x] https://github.com/EspressoSystems/HotShot/issues/2646
• [x] https://github.com/EspressoSystems/HotShot/issues/2647
• [x] https://github.com/EspressoSystems/HotShot/issues/2648
• [x] https://github.com/EspressoSystems/HotShot/issues/2649
• [x] https://github.com/EspressoSystems/HotShot/issues/2650
• [x] https://github.com/EspressoSystems/HotShot/issues/2731

Stretch goals

• [ ] https://github.com/EspressoSystems/HotShot/issues/2651
• [ ] https://github.com/EspressoSystems/HotShot/issues/2652

Are there any other details to include?

No response

What are the acceptance criteria to close this issue?

We should be able to both run a libp2p only network with these changes, and we should be able to see huge throughput speed increases when running with the fallback network.

Branch work will be merged to (if not the default branch)

No response