libp2p peerstore prevents gaining peers fast

[dapplion]

Describe the bug

Restarting a Lodestar beacon node after running for a while, it takes extremely long to gain peers to target_peers. We now that this doesn't happen on runs from scratch.

I did a test in two contabo nodes:

contabo-14: rm beacon/peer-id.json contabo-15: rm -r beacon/peerstore/

contabo-14: 2 min latter: 0 peers, 10 min latter 3 peers contabo-15: 2 min latter: 15 peers, 10 min latter 44 peers

Expected behavior

Having an existing peerstore should not delay gaining peers

Steps to Reproduce

• Run Lodestar in Prater without peerstore and time 0 to target_peers (should be < 5 min)
• Restart Lodestar and time 0 to target_peers (could be ~ 30 min)

contabo-14

contabo-15

[dapplion]

CC: @tuyennhv should we still go for https://github.com/ChainSafe/lodestar/compare/master...dapplion/ephemeral-peerid

[twoeths]

CC: @tuyennhv should we still go for master...dapplion/ephemeral-peerid

yeah I'm fine with creating new peer id upon restart (used to do that manually a lot of times), go ahead unless @wemeetagain has some information regarding discv5

[wemeetagain]

Should be fine

[dapplion]

Still observing this. The node appears to try to dial to the same peer over and over

[twoeths]

tested with v1.3.0/080b73b on a ctvpss node

1st time:

• Remove peerstore, node gets to targetPeers in 22 minutes
• Restart node, node gets to targetPeers in 9 minutes
• Restart node, node gets to targetPeers in 12 minutes

2nd time:

• Remove peerstore, node gets to targetPeers in 25 minutes
• Restart node, node gets to targetPeers in 23 minutes
• Restart node, node gets to targetPeers in 19 minutes

==> keeping peerstore even helps gaining peers a bit faster

Still observing this. The node appears to try to dial to the same peer over and over

fixed in https://github.com/ChainSafe/lodestar/pull/4969