Improve Gnosis keyper p2p connections

[jannikluhn]

In my testing setup, the Gnosis keypers don't create the gossiping mesh properly. This leads to high propagation times as messages are not gossiped immediately but only forwarded on request.

Logs

On startup, the keyper (12D3KooWRmoxBapB4EBZDK74R4yGwjL8uhgrPiRGRmNnbQuDMt7U) connects to the configured bootstrap node (12D3KooWMyutShWdqYj7fre4Vjuq2QnCTb26Ki1KpyDyVsrmKeki):

2024-03-01T11:38:29.675+0100	DEBUG	swarm2	swarm/swarm_dial.go:240	dialing peer	{"from": "12D3KooWRmoxBapB4EBZDK74R4yGwjL8uhgrPiRGRmNnbQuDMt7U", "to": "12D3KooWMyutShWdqYj7fre4Vjuq2QnCTb26Ki1KpyDyVsrmKeki"}

Soon after, they PRUNE the connection to it:

2024-03-01T11:38:29.776+0100	DEBUG	pubsub	[email protected]/gossipsub.go:849	PRUNE: Remove mesh link to 12D3KooWMyutShWdqYj7fre4Vjuq2QnCTb26Ki1KpyDyVsrmKeki in decryptionKeys

Later, they try to connect to another peer (12D3KooWLyURWEKUX9uGnx8wMTR7svUneR7GqE7SzRVU5Gs2cdk6) whose contact info they should have received from the bootstrap node during pruning. However, this fails:

2024-03-01T11:38:29.776+0100	DEBUG	pubsub	[email protected]/gossipsub.go:966	error connecting to 12D3KooWLyURWEKUX9uGnx8wMTR7svUneR7GqE7SzRVU5Gs2cdk6: failed to dial: failed to dial 12D3KooWLyURWEKUX9uGnx8wMTR7svUneR7GqE7SzRVU5Gs2cdk6: no addresses

It seems like the keyper did not receive the peer's address, even though they know the peer identity.

Effect

As a result of missing mesh peers, the propagation and subsequently key generation time is very large:

2024/03/01 12:39:07.095271 DBG [       keyper.go:349] sending decryption trigger block-number=24 num-identities=1 tx-pointer=0 tx-pointer-age=1
...
2024/03/01 12:39:08.152568 INF [messagingmiddleware.go:263] sending keys block=24 num-keys=1 num-signatures=2 tx-pointer=0

The time difference is ~1s, but should be on the order of milliseconds.

[jannikluhn]

It is to be expected that peer exchange only sends the peer id, but not the address. In this case, we are supposed to look up the peer in the DHT: https://github.com/libp2p/go-libp2p-pubsub/blob/048a4d30d0c3c00829181fd81aa697eb60497448/gossipsub.go#L1877-L1879

However, we disable the DHT: https://github.com/shutter-network/rolling-shutter/blob/8414b9e14582546be2fa81a074b9ff76c480050e/rolling-shutter/p2p/messaging.go#L88-L91

Simply setting those values to true does not do the trick unfortunately. Apparently, the DHT stays empty:

2024-03-01T18:38:56.924+0100	WARN	pubsub	[email protected]/discovery.go:207	bootstrap: error providing rendezvous for decryptionKeys: failed to find any peer in table

[jannikluhn]

Done