Persist known peers and use them on restart

[zah]

Since it's possible to implement this entirely on NBC's side by supplying a custom bootstrap nodes list on start-up, it looks like nim-eth should provide at most some optional helper modules for clients to use if they desire the feature.

[kdeme]

This issue is not 100% clear to me from its title and current description.

First, what are we attempting to achieve? Speeding up the connection to good peers on a restart of the beacon node or trying to be less reliant on bootstrap nodes? Or both?

One could track the most useful peers from different perspectives:

1. beacon chain
2. libp2p / gossipsub
3. discovery

I'm assuming from the current description that this issue is purely about case 3, which will provide better security and a (probably small) speed-up in start-up. Most speed-up would probably be achieved with 1 / 2.

In each case one could store peers based on some metric(s), these metrics might overlap but will not necessarily be the same. Easiest example, a bootstrap node might be highly available, great network access, uptime, etc., but it might be ONLY a discovery bootstrap node. Interesting for discovery, not for the other 2. However, this doesn't mean we do not want to store it, as this bootstrap node might provide us with a great set of nodes that are interesting for 1 and 2.

Next, when storing nodes for case 1 and 2, one will bypass discovery and directly try to dial these nodes. No need to send them to discovery as bootstrap nodes first.

For case 3 one would store these in a database based on discovery metrics (e.g. amount of successful request/responses), and next seed from these (together with bootstrap nodes) the discovery process. Relevant nim-eth issue https://github.com/status-im/nim-eth/issues/189. This can be implemented on NBC side, but I don't really see the benefit of this, as I don't believe we need to be e.g. mingling metrics from 1, 2 with 3, as the purposes are different. That doesn't mean of course that we can't nicely split this functionality from what there is now and make it optional, which is probably also what you mean with optional helper modules.

@cheatfate Please also chime in with your security considerations on storing this data.

[zah]

For now, I think we can focus this task to the 3rd case (discovery peers). The main goal is indeed to re-gain peers very quickly after a restart.

The feature can be implemented like this:

1. Subscribe to a shutdown event of the BeaconNode.
2. Within the handler, get all current peers from the discovery field of the Eth2Node object (defined in eth2_network.nim).
3. Write all the peers to the beacon_chain_db in a new dedicated table (see the usages of KvStoreRef, openKvStore and prepareStmt). The records should include the ENR plus a timestamp indicating when it was written.
4. On start-up clean up all persisted ENR records that are older than EnrRetention (a newly introduced constant)
5. Use the remaining records as bootstrap nodes (see the usages of newProtocol in eth2_discovery.nim). Please note that the existing mechanism of allowing a bootstrap_nodes.txt file in the data directory of the client is intended for bootstrap nodes that are manually managed by the user. The mechanism described in this message is about a more automated persistence scheme that's always enabled and used behind the scenes.