Support for responding to a LocalStateQuery demand for a snapshot of big ledger peers

[crocodile-dentist]

trafficstars

This change equips LocalStateQuery mini protocol app to respond to a request for a snapshot, taken from the current tip, of big ledger peers from a client, eg. cardano-cli. A new query tag GetBigLedgerPeerSnapshot is added to BlockQuery. This data can be loaded later by a node, and it may be leveraged by the network layer to facilitate syncing up a node from a blank or stale state. Diffusion layer is expected to make heavy use of these relays when bootstrapping from scratch in Genesis consensus mode.

[crocodile-dentist]

I've looked into the performance impact on cpu and memory use of this query on a fully synced node without any peers connected, and results are as follows. Measurements are taken every second for a duration of 5 minutes on an [email protected] GHz with 32 GB of RAM.

5 minute baseline plot when idling:

5 minutes of snapshot requests, where to node is queried in random intervals every 0 to 1 seconds:

There isn't any discernable impact on memory use - no spikes or significant/unusual growth could be determined, and any memory use triggered by the query is neglibigle. Some minor CPU use in excess of the idling load is visible on the second plot, but nothing noteworthy can be concluded from it as it appears to be within reasonable limits.

The query itself executes in around 20ms on the node: 50-100 us to acquire ledger state in an atomic operation <3 ms to calculate stake pool relays by getPeers (time complexity~ O(n log n) - almost, there is a square term due to removal of duplicate relays for each stake pool, but I assume that the number of relays for each pool is small and can be treated as a constant to simplify things) <15 ms to calculate which are the big ledger peers (time complexity is O(n log n) with most likely bigger constant terms as there are multiple passes over the stake pools, eg. one pass to calculate the total stake and another to re-compute the relative stake of each pool, and another to sort things)

The resulting snapshot file size is ~50kb.

[jasagredo]

For the fs-sim error I see in ci, update to the CHaP index-state at 2024-06-06T15:28:08Z

[crocodile-dentist]

For the fs-sim error I see in ci, update to the CHaP index-state at 2024-06-06T15:28:08Z

Yes, that's what I figured

[amesgen]

Ah, and the new golden files have to be checked in (that's why CI is red ATM). You can generate them locally via cabal build all and then cabal test all --test-options='-p Golden'.