Unexpected behavior when trying to connect to bootstrap nodes

[dvmazur]

We want to set up our custom network so that machines under NAT can still connect to each other. We tried providing our bootstrap nodes, but they didn't return the public address of our under-NAT machines.

Then we tried providing nodes from this go-libp2p example and encountered some strange behavior: when providing only one of the peers nothing works (either we can't connect, or the peer doesn't return the public IP), but when providing both everything works just fine.

What could be the issue? Could the issue be with how we configure our nodes? If yes, what options do we need to specify, so the peers return the public addresses of other nodes?

[vyzo]

Seems you are announcing 0.0.0.0 addresses with the -hostAddrs option.

[vyzo]

And you are also only listening on localhost. That's definitely not what you want.

[MaximKsh]

-listen address is using for connecting to daemon from binding, isn't it? It may be unix socket as well. Do we (with @deniskamazur ) misunderstand something? What is the proper way to start p2pd node with public IP and several nodes under NAT so that "under NAT" nodes acquire "behind NAT" IP and may connect to each other?

[vyzo]

Sorry, my bad -- you are right. I haven't touched this code in a while and I forget what all the options do :)

[vyzo]

Your addrs options are perfectly fine, sorry for my misleading comments.

[vyzo]

ok, so your bootstrappers are using tcp and you are listening on quic; I wonder whether this is a problem in this case.

[vyzo]

which shouldn't be because it is also using the default transports.

[MaximKsh]

I describe task detailed. We have node with public IP (node A):

$ ./p2pd -tls=1 -connManager=1 -natPortMap=1 -autonat=1 -dhtServer=1   -hostAddrs=/ip4/0.0.0.0/tcp/5000
Control socket: /unix/tmp/p2pd.sock
Peer ID: QmUN5jQC26K3FM3XSjjPcKBWZAaxB9NiQBcL3XZPhaWvPX
Peer Addrs:
/ip4/92.53.78.235/tcp/5000
/ip4/127.0.0.1/tcp/5000

and node under NAT (node B) with bootstrapping to node A:

$ ./p2pd -tls=1 -connManager=1 -natPortMap=1 -autonat=1 -dhtClient=1 -b -bootstrapPeers=/ip4/92.53.78.235/tcp/5000/p2p/QmUN5jQC26K3FM3XSjjPcKBWZAaxB9NiQBcL3XZPhaWvPX -hostAddrs=/ip4/0.0.0.0/tcp/44002
Control socket: /unix/tmp/p2pd.sock
Peer ID: Qmdn5vaZ3tR81wfDVQ5m1Z4dWmKXUFwuhPNGvwf5FYM1CM
Peer Addrs:
/ip4/192.168.1.150/tcp/44002
/ip4/127.0.0.1/tcp/44002
Bootstrap peers:
/ip4/92.53.78.235/tcp/5000/p2p/QmUN5jQC26K3FM3XSjjPcKBWZAaxB9NiQBcL3XZPhaWvPX

Peer addrs list contains only LAN addresses. But if connect to global IPFS network node B has "behind NAT" IP:

$ ./p2pd -tls=1 -connManager=1 -natPortMap=1 -autonat=1 -dhtClient=1 -b -hostAddrs=/ip4/0.0.0.0/tcp/44002
Control socket: /unix/tmp/p2pd.sock
Peer ID: Qmdp341BhnUdZxxPU873qP6NtYT1DrdMGNixG9S41wJKfL
Peer Addrs:
/ip4/192.168.1.150/tcp/44002
/ip4/127.0.0.1/tcp/44002
/ip4/109.252.14.143/tcp/19973   <--- behind NAT IP. Why it shows only if connect to IPFS network?

How to run node A (public IP machine) so that node B gets its "behind NAT" ?

[vyzo]

This is because there needs to be some confidence on the observations in autonat; you need to connect to enough peers to verify that (I think it's 3 or 4), so you can't get it from a single bootstrapper.

We could expose the ForceReachability option in go-libp2p, should be simple to implement if you care for a patch.

[dvmazur]

Thanks for your help!

[dvmazur]

The ForceReachability option actually did not solve our problem as running only one bootstrap doesn't return the public IP