netbird icon indicating copy to clipboard operation
netbird copied to clipboard

Published 20 hours ago • verified publisher icon netbirdio

client can't use p2p connection

Open SamPeng87 opened this issue 4 months ago • 1 comments

Describe the problem First, I have a routeros as the main router. The main configuration features are as follows:

  • Hairpinning NAT, because the STUN domain needs to be accessed from the internal network
  • NAT1, routeros only supports UDP, and I’m not sure if this affects netbird

Secondly, I have 3 VLANs. Each of the first two VLANs has a netbird client, and the other VLAN has a netbird self-hosted deployment in Docker. Yes, I have deployed netbird’s STUN and client within the internal network because I don’t have a VPS to do this.

The current issue is:

The client on the internal network shows the public IP as the internal IP 192.168.xx.xx. I am not sure if this has any impact; the connection status between them is P2P. This is good. However, any device connecting to the peer from the external network shows as Relayed. I have been trying for a whole day.

There is only one situation where it improves: In the coturn server configuration, setting --external-ip="$(detect-external-ip)/priverip" is not practical because my external IP changes frequently. I have a dynamic external IP. The docker cmd is static.

I don't know where the problem is or how to troubleshoot it. Checking the debug logs. PIONS_LOG_DEBUG=all NB_LOG_LEVEL=debug netbird up -F

can't connect by p2p

2025-06-23T17:20:56+08:00 INFO [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/handshaker.go:89: received connection confirmation, running version 0.47.1 and with remote WireGuard listen port 51820
2025-06-23T17:20:56+08:00 INFO [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/handshaker.go:77: wait for remote offer confirmation
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:82: OnNewOffer for ICE
2025-06-23T17:20:56+08:00 DEBG relay/client/manager.go:143: open peer connection via permanent server: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_relay.go:68: handled offer by reusing existing relay connection
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:98: recreate ICE agent
2025-06-23T17:20:56+08:00 INFO client/internal/peer/ice/env.go:48: setting ICE disconnected timeout to 30 seconds
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:109: gather candidates
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:119: turn agent dial
ice DEBUG: 17:20:56.712477 agent.go:399: Started agent: isControlling? false, remoteUfrag: "KRPbHwIEAJERcQwb", remotePwd: "abbhhqaeHSsEkzBjAVwkaKpkvgzsrOnk"
ice INFO: 2025/06/23 17:20:56 Setting new connection state: Checking
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:212: ICE ConnectionState has changed to Checking
ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet.
ice WARNING: 2025/06/23 17:20:56 UDPMuxDefault should not listen on unspecified address, use NewMultiUDPMuxFromPort instead
ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet.
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:300: discovered local candidate udp4 host 0.0.0.0:51823
ice DEBUG: 17:20:56.713625 agent.go:820: Ignore duplicate candidate: udp4 srflx <public ip>:9168 related 0.0.0.0:51823
ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet.
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:300: discovered local candidate udp4 srflx <public ip>:9168 related 0.0.0.0:51823
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 172.21.128.1:51820
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp6 host [2408:8207:8435:2160:483a:e036:d745:883c]:51820
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 172.21.128.1:51820 for bSycnEodzARkiKEw
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 172.21.128.1:51820 for bSycnEodzARkiKEwstun:home.isampeng.net:3478
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 192.168.10.254:51820
2025-06-23T17:20:56+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 host 192.168.10.254:51820], its address is part of routed network 192.168.10.0/24
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp6 host [2408:8207:8435:2160:80c8:370c:6ba7:baba]:51820
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 127.0.0.1:51820
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 srflx 192.168.10.254:51820 related 0.0.0.0:51820
2025-06-23T17:20:56+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 srflx 192.168.10.254:51820 related 0.0.0.0:51820], its address is part of routed network 192.168.10.0/24
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 127.0.0.1:51820 for bSycnEodzARkiKEw
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 127.0.0.1:51820 for bSycnEodzARkiKEwstun:home.isampeng.net:3478
2025-06-23T17:20:58+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:578: send offer to peer
2025-06-23T17:20:58+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:282: OnRemoteAnswer, priority: PriorityRelay, status ICE: Disconnected, status relay: Connected

can connection by p2p

025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [2408:8207:8435:2160:10cb:3eb8:db30:5470]:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.10.253:51821
2025-06-23T17:27:47+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 host 192.168.10.253:51821], its address is part of routed network 192.168.10.0/24
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 198.19.249.3:51821
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 198.19.249.3:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.97.0:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [2408:8207:8435:2160:25ce:fa51:ec1e:df84]:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [fd07:b51a:cc66:0:a617:db5e:ab7:e9f1]:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.107.0:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 srflx 124.64.234.240:51821 related 0.0.0.0:51821
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 198.19.249.3:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.97.0:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.97.0:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.107.0:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.107.0:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 124.64.234.240:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 124.64.234.240:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:531: start to communicate with peer via relay
2025-06-23T17:27:47+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/lazyconn/manager/manager.go:496: peer connected, pausing inactivity monitor while connection is not disconnected
2025-06-23T17:27:48+08:00 INFO [peer: UqVSMbHF+9DZE7rrc6vcgjgS4SD75kgfSyy4SJ7sjiM=] client/internal/peer/guard/guard.go:74: start reconnect loop...
2025-06-23T17:27:48+08:00 INFO [peer: o3ZzcLJu2RU/WwxWzF6/LeOLCEJr5WZqF6OjSbSRBF8=] client/internal/peer/guard/guard.go:74: start reconnect loop...
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:212: ICE ConnectionState has changed to Connected
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:125: agent dial succeeded
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:330: selected candidate pair [local <-> remote] -> [udp4 srflx <public ip>:9168 related 0.0.0.0:51823 <-> udp4 srflx 124.64.234.240:51821 related 0.0.0.0:51821], peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:154: on ICE conn is ready to use
2025-06-23T17:27:48+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:359: set ICE to active connection
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/wg_watcher.go:80: disable WireGuard watcher
2025-06-23T17:27:48+08:00 DEBG client/iface/iface.go:120: updating interface utun100 peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=, endpoint 124.64.234.240:51821, allowedIPs [100.116.237.209/32]
2025-06-23T17:27:48+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/wg_watcher.go:112: WireGuard watcher stopped

I suspect it is caused by a routing issue because last week I was using a non-routeros as the main router, which was a very ordinary unmanaged router, but when I connected through that router, P2P worked normally. After changing the router, it stopped working. The network topology hasn't changed much. I have tried routeros many times but haven't found any good solution.

To Reproduce

Steps to reproduce the behavior:

Expected behavior

all outside client can use p2p connection type

Are you using NetBird Cloud?

self-host NetBird's control plane.

NetBird version

0.47

Is any other VPN software installed?

no

Debug output

To help us resolve the problem, please attach the following anonymized status output

netbird status -dA

Peers detail:
 desktop-mvk81eo.anon-E3Ef7.domain:
  NetBird IP: 100.116.34.5
  Public key: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 8 seconds ago
  Last WireGuard handshake: 1 minute, 39 seconds ago
  Transfer status (received/sent) 408 B/616 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

 friendlywrt.anon-E3Ef7.domain:
  NetBird IP: 100.116.65.72
  Public key: o3ZzcLJu2RU/WwxWzF6/LeOLCEJr5WZqF6OjSbSRBF8=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 15 seconds ago
  Last WireGuard handshake: 15 seconds ago
  Transfer status (received/sent) 665.6 KiB/145.2 KiB
  Quantum resistance: false
  Networks: 192.168.10.0/24
  Latency: 0s

 iphone-admin.anon-E3Ef7.domain:
  NetBird IP: 100.116.146.43
  Public key: UqVSMbHF+9DZE7rrc6vcgjgS4SD75kgfSyy4SJ7sjiM=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 15 seconds ago
  Last WireGuard handshake: 5 seconds ago
  Transfer status (received/sent) 308 B/860 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

 penglingjundemac-mini.anon-E3Ef7.domain:
  NetBird IP: 100.116.237.209
  Public key: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=
  Status: Connected
  -- detail --
  Connection type: P2P
  ICE candidate (Local/Remote): srflx/srflx
  ICE candidate endpoints (Local/Remote): 198.51.100.0:9168/198.51.100.1:51821
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 8 seconds ago
  Last WireGuard handshake: 9 seconds ago
  Transfer status (received/sent) 41.3 KiB/33.2 KiB
  Quantum resistance: false
  Networks: -
  Latency: 11.088333ms

Events:
  [INFO] SYSTEM (963dbb96-e643-4004-ab2c-1f4cf7d546f0)
    Message: Network map updated
    Time: 4 minutes, 16 seconds ago
OS: darwin/arm64
Daemon version: 0.47.1
CLI version: 0.47.1
Management: Connected to https://home.anon-gHZ82.domain:1203
Signal: Connected to https://home.anon-gHZ82.domain:1203
Relays:
  [stun:home.anon-gHZ82.domain:3478] is Available
  [turn:home.anon-gHZ82.domain:3478?transport=udp] is Available
  [rels://home.anon-gHZ82.domain:1203] is Available
Nameservers:
FQDN: penglingjundemac-studio-7.anon-E3Ef7.domain
NetBird IP: 100.116.205.173/16
Interface type: Userspace
Quantum resistance: false
Lazy connection: true
Networks: -
Forwarding rules: 0
Peers count: 4/4 Connected

Create and upload a debug bundle, and share the returned file key:

netbird debug for 1m -AS -U

use external-ip ,has one p2p 8f231557c3b2a09fc122090c8738b19a618321b016d4b5bf41def293d760b7c0/7d8079fb-0c1c-4a3a-86d6-9a3bf1aa0bd4

remove external-ip,no any p2p 8f231557c3b2a09fc122090c8738b19a618321b016d4b5bf41def293d760b7c0/14de4869-6f0e-4e6b-8f40-b71694e92800

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

Have you tried these troubleshooting steps?

  • [x] Reviewed client troubleshooting (if applicable)
  • [x] Checked for newer NetBird versions
  • [x] Searched for similar issues on GitHub (including closed ones)
  • [x] Restarted the NetBird client
  • [x] Disabled other VPN software
  • [x] Checked firewall settings

SamPeng87 avatar Jun 23 '25 09:06 SamPeng87