netbird icon indicating copy to clipboard operation
netbird copied to clipboard

Abuse complaint from ISP: NetBird client sending unsolicited traffic to CGNAT subnet

Open mrchupatek opened this issue 9 months ago • 11 comments

Describe the problem

We received an abuse complaint from our ISP due to NetBird client sending unsolicited UDP and TCP traffic to IPs in the 100.79.0.0/16 CGNAT subnet. The ISP flagged this as a network scan attempt.

This subnet is used internally by NetBird on interface wt0, but external probing of CGNAT addresses appears suspicious from the ISP's point of view and may lead to service suspension.


To Reproduce

  1. Install NetBird (default setup, no configuration changes)
  2. Join a self-hosted network where 100.79.0.0/16 is assigned to wt0
  3. Observe outbound connections to 100.79.x.x addresses
  4. Receive abuse message from provider reporting "Netscan" activity

Expected behavior

NetBird should avoid initiating connections to RFC6598 (CGNAT) IP ranges unless explicitly allowed. At a minimum, this behavior should be configurable.


Are you using NetBird Cloud?

No, using self-hosted NetBird.


NetBird version

Daemon version: 0.44.0 CLI version: 0.44.0


Is any other VPN software installed?

No.


Firewall (UFW) settings

# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
Anywhere                   ALLOW IN    100.64.0.0/10             
Anywhere                   ALLOW IN    192.168.0.0/24            
Anywhere                   ALLOW IN    46.101.102.23             

10.0.0.0/8                 DENY OUT    Anywhere on eno1          
172.16.0.0/12              DENY OUT    Anywhere on eno1          
192.168.0.0/16             DENY OUT    Anywhere on eno1

Abuse

TIME (UTC)           SRC             SRC-PORT -> DST             DST-PORT SIZE PROT
-------------------------------------------------------------------------------
2025-05-26 07:35:35  37.27.133.173  8301     -> 100.79.0.94      8301     215 UDP
2025-05-26 07:35:35  37.27.133.173  8301     -> 100.79.2.178     8301     215 UDP
2025-05-26 07:35:25  37.27.133.173 50222     -> 100.79.3.2       8300      78 TCP
2025-05-26 07:35:26  37.27.133.173 50222     -> 100.79.3.2       8300      78 TCP
2025-05-26 07:35:24  37.27.133.173 50222     -> 100.79.3.2       8300      78 TCP
2025-05-26 07:35:14  37.27.133.173  8301     -> 100.79.3.2       8301     141 UDP
2025-05-26 07:35:30  37.27.133.173  8301     -> 100.79.3.2       8301     243 UDP
2025-05-26 07:35:31  37.27.133.173 54762     -> 100.79.3.2       8301      78 TCP
2025-05-26 07:34:46  37.27.133.173  8301     -> 100.79.3.62      8301     150 UDP
2025-05-26 07:35:06  37.27.133.173  8301     -> 100.79.3.62      8301     144 UDP
2025-05-26 07:35:15  37.27.133.173  8301     -> 100.79.4.71      8301     266 UDP

Additional context

  1. We use the default NetBird configuration out-of-the-box, no modifications.
  2. The internal network is assigned the 100.79.0.0/16 subnet on interface wt0.
  3. UFW explicitly blocks outbound access to other private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), but not CGNAT, as this is needed for NetBird.
  4. The packets appear to be part of peer discovery or connection attempts, but to the provider this behavior mimics a port scan.

mrchupatek avatar May 26 '25 08:05 mrchupatek

@mrchupatek thanks for reporting the issue. Can you confirm the version of all peers connecting to the device(s) behind the ISP?

mlsmaycon avatar May 26 '25 09:05 mlsmaycon

@mlsmaycon we have different versions of netbird client from 0.42 to latest . This problem is not fixed on all servers where there is netbird

mrchupatek avatar May 26 '25 09:05 mrchupatek

Is the problem fixed, or does it happen on all servers?

If it is still happening, can you send us the bundle file generated by the following command on two nodes (remote and from this ISP):

netbird debug for 1m -AS -U

If the upload bundle is not supported, just remove the -U flag and upload the files here.

mlsmaycon avatar May 26 '25 09:05 mlsmaycon

The problem still exists. Netbird tries to send data via external interface, but on private network. I am attaching two debug files from different servers where the problem was detected.

netbird.debug.287350374.zip netbird.debug.862272334.zip

mrchupatek avatar May 26 '25 10:05 mrchupatek

@mlsmaycon Hi, this issue mostly occurs with Hetzner. They have a scanner detection system, and when using the Netbird client (any version), we receive abuse reports stating that we're scanning local networks through the external interface.

As a temporary fix, we block the following via iptables:

10.0.0.0/8 DENY OUT Anywhere on eno1
172.16.0.0/12 DENY OUT Anywhere on eno1
192.168.0.0/16 DENY OUT Anywhere on eno1

We’d appreciate a solution on your side that prevents the Netbird client from scanning local networks through the external interface.

jack-shax avatar May 26 '25 14:05 jack-shax

Hi @mrchupatek,

I cannot find evidence in the bundles that this is coming from NetBird, unless, for some reason, the issue didn't occur in the time frame when you ran the debug command.

2025-05-26 07:35:35  37.27.133.173  8301     -> 100.79.0.94      8301     215 UDP
2025-05-26 07:35:35  37.27.133.173  8301     -> 100.79.2.178     8301     215 UDP
2025-05-26 07:35:25  37.27.133.173 50222     -> 100.79.3.2       8300      78 TCP
2025-05-26 07:35:26  37.27.133.173 50222     -> 100.79.3.2       8300      78 TCP
2025-05-26 07:35:24  37.27.133.173 50222     -> 100.79.3.2       8300      78 TCP
2025-05-26 07:35:14  37.27.133.173  8301     -> 100.79.3.2       8301     141 UDP
2025-05-26 07:35:30  37.27.133.173  8301     -> 100.79.3.2       8301     243 UDP
2025-05-26 07:35:31  37.27.133.173 54762     -> 100.79.3.2       8301      78 TCP
2025-05-26 07:34:46  37.27.133.173  8301     -> 100.79.3.62      8301     150 UDP
2025-05-26 07:35:06  37.27.133.173  8301     -> 100.79.3.62      8301     144 UDP
2025-05-26 07:35:15  37.27.133.173  8301     -> 100.79.4.71      8301     266 UDP

This doesn't look like NetBird traffic, NetBird uses (if not changed) udp/51820 on either side. The listed ports or tcp are not in use. Could it be another application that erroneously uses NetBird's interface IP address?

UFW explicitly blocks outbound access to other private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), but not CGNAT, as this is needed for NetBird.

You can additionally block the CGNAT range on the physical interface or SNAT it. The range in use by NetBird is not supposed to leave the overlay network unless specifically routed elsewhere.

@jack-shax

As a temporary fix, we block the following via iptables:

10.0.0.0/8 DENY OUT Anywhere on eno1 172.16.0.0/12 DENY OUT Anywhere on eno1 192.168.0.0/16 DENY OUT Anywhere on eno1

We’d appreciate a solution on your side that prevents the Netbird client from scanning local networks through the external interface.

Unfortunately NetBird cannot know which networks are behind which interface, so we need to probe all candidate IPs. We could implement some filter, but that wouldn't be different than firewalling the subnets off like you do.

lixmal avatar May 26 '25 14:05 lixmal

@lixmal Hi. The problem is that on a physical server we just rented — where only the Netbird client was installed (we're using a self-hosted setup) — we received an abuse report from the provider about 10 minutes after connecting the client to the management server. The report states that we're scanning local network ranges via the external interface.

Nothing else was configured on the server — only the Netbird client was running.

jack-shax avatar May 26 '25 14:05 jack-shax

@jack-shax is the IP 37.27.133.173 above from one of your nodes?

mlsmaycon avatar May 26 '25 15:05 mlsmaycon

@mlsmaycon yes

jack-shax avatar May 26 '25 17:05 jack-shax

Sorry @jack-shax , my question was incomplete. This node is a peer within NetBird, right? not a service like netbird management, coturn, signal or relay.

mlsmaycon avatar May 26 '25 17:05 mlsmaycon

@mlsmaycon Yes, that’s correct. This is a node (client) in the Netbird network.

jack-shax avatar May 26 '25 17:05 jack-shax

I now have been hit with the same issue, This has caused not only our pirmary but backup firewall to get blocked. Before removing the affected Ubuntu Server, it was noted the only installed service was indeed netbird.

TIME (UTC) SRC SRC-PORT -> DST DST-PORT SIZE PROT

2025-08-04 19:12:39 176.x.x.197 19010 -> 100.114.156.65 10050 74 TCP 2025-08-04 19:12:40 1176.x.x.197 19010 -> 100.114.156.65 10050 74 TCP 2025-08-04 19:12:39 176.x.x.197 46462 -> 100.114.156.66 10050 74 TCP 2025-08-04 19:12:40 176.x.x.197 46462 -> 100.114.156.66 10050 74 TCP 2025-08-04 19:12:39 176.x.x.197 13453 -> 100.114.156.67 10050 74 TCP 2025-08-04 19:12:40 176.x.x.197 13453 -> 100.114.156.67 10050 74 TCP 2025-08-04 19:12:39 176.x.x.197 39620 -> 100.114.156.68 10050 74 TCP 2025-08-04 19:12:40 176.x.x.197 39620 -> 100.114.156.68 10050 74 TCP 2025-08-04 19:12:39 176.x.x.197 65518 -> 100.114.156.69 10050 74 TCP 2025-08-04 19:12:40 176.x.x.197 65518 -> 100.114.156.69 10050 74 TCP 2025-08-04 19:12:39 176.x.x.197 62427 -> 100.114.156.70 10050 74 TCP 2025-08-04 19:12:40 176.x.x.197 62427 -> 100.114.156.70 10050 74 TCP 2025-08-04 19:12:40 176.x.x.197 55156 -> 100.114.156.71 10050 74 TCP 2025-08-04 19:12:39 176.x.x.197 55156 -> 100.114.156.71 10050 74 TCP 2025-08-04 19:12:40 176.x.x.197 52593 -> 100.114.156.72 10050 74 TCP

SLH-Skyler avatar Aug 05 '25 13:08 SLH-Skyler

I think we can close this, because account's IP range is now customizable after https://github.com/netbirdio/netbird/releases/tag/v0.53.0 got released

Image

Feel free to comment if you believe otherwise and we'll reopen.

nazarewk avatar Aug 06 '25 12:08 nazarewk