[RFC] limiter: smarter perPeerLimit

[magik6k]

This makes the limiter.perPeerLimit depend on fdConsuming.

graph of perPeerLimit vs fdConsuming:

[magik6k]

(Will work on fixing tests once the idea is approved)

[Stebalien]

So, this looks like it decreases per-peer dials as we run out of file descriptors? We should make sure this doesn't lead to a situation where everything is slow and therefore none of the dials really finish.

[bigs]

i think this change should hold off until we get a better idea of where we're getting bogged down when dialing DHT nodes

[raulk]

This idea is interesting. Choking the peer limit as the FD allowance runs out. However, I worry that this will bring unfairness to the system. Also won't work well with the address pollution problem of the DHT (e.g. we have peers with 200 addresses), unless we implement an address scoring system. Right now we're pretty dumb and dial addresses in the order returned by the peerstore, but if we can prioritise certain addresses, even in a choked scenario we could end up dialling successfully.

FYI @magik6k – we traced down the source of many woes to the DHT tripping over the FD limit in the swarm, and we are now pacing dials dynamically: https://github.com/libp2p/go-libp2p-kad-dht/pull/237.

Are you still interested in discussing ideas around this?