Unable to connect via P2P even public IPv4 is available

[Integral-Tech]

Describe the problem

I am pretty sure that one of my peers has public IPv4 address, however I fail to connect two peers via P2P.

• Peer 1: Arch Linux (without public IPv4)
• Peer 2: OpenWrt (with public IPv4)

To Reproduce

Steps to reproduce the behavior:

1. Start Netbird service on two peers (one of them has public IPv4 address)
2. Run netbird status -d

Expected behavior

  Status: Connected
  -- detail --
  Connection type: P2P
  Direct: true

Are you using NetBird Cloud?

Yes

NetBird version

0.28.9

NetBird status -dA output:

Peers detail:
 *.netbird.cloud:
  NetBird IP: *
  Public key: *
  Status: Connected
  -- detail --
  Connection type: Relayed
  Direct: true
  ICE candidate (Local/Remote): host/relay
  ICE candidate endpoints (Local/Remote): *
  Last connection update: 14 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 444 B/424 B
  Quantum resistance: false
  Routes: 192.168.0.0/16
  Latency: 426.126669ms

OS: linux/amd64
Daemon version: 0.28.9
CLI version: 0.28.9
Management: Connected to https://api.netbird.io:443
Signal: Connected to https://signal.netbird.io:443
Relays: 
  [stun:stun.netbird.io:5555] is Available
  [turns:turn.netbird.io:443?transport=tcp] is Available
Nameservers: 
FQDN: *.netbird.cloud
NetBird IP: *
Interface type: Kernel
Quantum resistance: false
Routes: -
Peers count: 1/1 Connected

[pascal-fischer]

Hi @Integral-Tech, to have a full P2P connection both peers need to be reachable directly. In your status output you see ICE candidate (Local/Remote): host/relay so one way is connected on host level (due to the public IP) which would allow P2P but the opposite direction is relayed. Thats why overall the connection type is listed as Relayed

[Marcus1Pierce]

@Integral-Tech From what I know, if you want both peers to communicate via P2P, both peers must be on the same IP segment, or both peers must have a public IP and be accessible from outside.

#CMIIW

[Spiritreader]

Hi @Integral-Tech, to have a full P2P connection both peers need to be reachable directly. In your status output you see ICE candidate (Local/Remote): host/relay so one way is connected on host level (due to the public IP) which would allow P2P but the opposite direction is relayed. Thats why overall the connection type is listed as Relayed

If one direction can connect via host, isn't it possible to just establish a direct wireguard connection? This is essentially the base scenario of manually setting up a wireguard tunnel for machines behind CGNAT:

• Open whatever port (12345) on PeerA.
• Configure wireguard on CTNATed PeerB to add PeerA with the address of PeerA:12345.

Now, as long as there is a keepalive interval to prevent the CGNAT's stateful FW from closing the connection, you're good to go. I had to do this fairly often for peers before I used netbird, as some of my devices are behind CGNAT.

Having host/relay, kinda sounds like calling a friend on his phone via the number they gave you, they pick up and say "Hello" and in order to respond you send your grandma to his house via bike.

if we had a TCP connection on port 80 for CGNATed devices, some form of return traffic must be allowed directly because we need some form of traffic to flow between devices if they have a connection to each other.

Or am I understanding sth fundamentally wrong here, for example that the ICE candidate doesn't show an established connection, but rather only the peer addresses?

[vampywiz17]

Hi @Integral-Tech, to have a full P2P connection both peers need to be reachable directly. In your status output you see ICE candidate (Local/Remote): host/relay so one way is connected on host level (due to the public IP) which would allow P2P but the opposite direction is relayed. Thats why overall the connection type is listed as Relayed

If one direction can connect via host, isn't it possible to just establish a direct wireguard connection? This is essentially the base scenario of manually setting up a wireguard tunnel for machines behind CGNAT:

• Open whatever port (12345) on PeerA.
• Configure wireguard on CTNATed PeerB to add PeerA with the address of PeerA:12345.

Now, as long as there is a keepalive interval to prevent the CGNAT's stateful FW from closing the connection, you're good to go. I had to do this fairly often for peers before I used netbird, as some of my devices are behind CGNAT.

Having host/relay, kinda sounds like calling a friend on his phone via the number they gave you, they pick up and say "Hello" and in order to respond you send your grandma to his house via bike.

if we had a TCP connection on port 80 for CGNATed devices, some form of return traffic must be allowed directly because we need some form of traffic to flow between devices if they have a connection to each other.

Or am I understanding sth fundamentally wrong here, for example that the ICE candidate doesn't show an established connection, but rather only the peer addresses?

This is how it work Tailscale. not need to "active" both client, it enought that only one is active.

[etranger7]

Hi, I have a similar problem. I am fairly certain my firewall configuration is correct. In fact, before I reset my firewall, I had a P2P connection. Unless I'm missing something, I reinstated the same rules but this time I'm getting ICE candidate (Local/Remote): srflx/relay instead of a P2P connection. @Integral-Tech were you able to figure this out?

[nazarewk]

@etranger7 could you send us a debug bundle (without anonymization -A) for ~5 minutes to review how the connectivity is being established between the peers? Also, please state which peers exactly those are. You can mail it to support at nebird.io or send directly to me on Slack (kdn).

I have checked myself with one host (my laptop) running behind CGNAT (not my regular ISP) and the other one having a dedicated public IPv4 (Hetzner VM), and the connectivity is still established with P2P.

[etranger7]

Hi, thanks for looking into this. Despite not changing anything in the firewall settings, the connection went back to P2P. I forgot to mention, we're testing this self-hosted. There seems to be some confusion over how peers make connections to the server and other peers when they are behind a firewall. I went through the documentation and I understand the general principles of signaling and ICE. However, I don't quite understand some specifics.

For example. Let's assume we have a peer behind a firewall. The firewall allows all outbound connections. There are no specific NAT rules pointing to this peer. This peer should still be able to make P2P connections to other peers outside of that LAN by making an outbound connection, to both the management server and the other peer, isn't it?