Zerotier creates huges amouts of UPnP port mappings and crashes the router

[DontBreakAlex]

Zerotier is basically unusable for me. As you can see on the screenshot, it constantly creates UPnP port mappings with result in the router crashing after 1 or 2 hours.

I am using zerotier 1.10.6 on debian 11.

This may or may not be a bug in zerotier but I can't imagine my ISP fixing its router if it is a bug in their UPnP implementation.

Let me know if you need more info.

[laduke]

you can disable upnp with the portMappingEnabled option here: https://github.com/zerotier/ZeroTierOne/tree/dev/service

Haven't seen or heard about this from other folks.

[DontBreakAlex]

Thanks, will do. Is there a way to set the external port to some manual value so that entries are not duplicated but UPnP ? What is the performance cost of disabling UPnP ?

[laduke]

Is there a way to set the external port to some manual value so that entries are not duplicated but UPnP ?

I don't think so.

What is the performance cost of disabling UPnP ?

Behind a ISP router, you probably don't need UPnP. UDP hole punching has a good chance of working without it. ipv6 helps with direct connections too.

[Tha14]

I also believe that zerotier one creates way too many upnp mappings but I am not sure how many are actually needed for proper operation.

[laduke]

only 1!

[Tha14]

only 1!

I am seeing way more than one. It's either that it creates too many or it creates a new one each time without expiring the older ones.

EDIT: Using the windows 10 build v1.10.5.

[laduke]

I can't reproduce this on my routers so it's hard to guess what could be happening.

If anyone is able to compile zerotier, give it a try. It looks like you have to un-comment #define ZT_PORTMAPPER_TRACE 1 in PortMapper.cpp

and then make, sudo ./zerotier-one (stop your installed/system zerotier-one first!)

The code is here https://github.com/zerotier/ZeroTierOne/blob/dev/osdep/PortMapper.cpp

[Tha14]

I can't reproduce this on my routers so it's hard to guess what could be happening.

If anyone is able to compile zerotier, give it a try. It looks like you have to un-comment #define ZT_PORTMAPPER_TRACE 1 in PortMapper.cpp

and then make, sudo ./zerotier-one (stop your installed/system zerotier-one first!)

The code is here https://github.com/zerotier/ZeroTierOne/blob/dev/osdep/PortMapper.cpp

Sorry to bother you about this but I've made the required change and build the service for windows. I am starting the service on the console now and I'm not sure where the trace log is being saved. I saw that there is data being written into metrics.prom but I'm not sure if that would help you. Please advise. Thanks

[laduke]

It should show in the same console where you start zerotier. Hrm

It should look something like this:

PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366b      
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366a     
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e6133669        
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:Layer3Forwarding:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANPPPConnection:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANCommonInterfaceConf
...

if you already disabled portmapping in your local.conf, maybe you won't see it...

[Tha14]

It should show in the same console where you start zerotier. Hrm

It should look something like this:

PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366b      
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366a     
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e6133669        
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:Layer3Forwarding:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANPPPConnection:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANCommonInterfaceConf
...

if you already disabled portmapping in your local.conf, maybe you won't see it...

I'm on windows. I'm not sure where local.conf is located so I am still using the default configuration. It's definitely not in my programdata folder. EDIT: Ignore my replies. My problem seems to be irrelevant to what the OP has posted.

[Nevcairiel]

I'm also seeing a new port mapping being generated anytime ZT1 starts up, quickly accumulating over time. Its not finding and re-using the old one, or not clearing the old one in some conditions.

I have turned off port mapping in the meantime, as it seems to work just fine as is.

[laduke]

https://docs.zerotier.com/zerotier/zerotier.conf has the locations of local.conf for various operating systems.

Let us know if you're still seeing the port mapping issues on versions 1.12.1 or later.

It's not impossible it's not fixed, but we can't reproduce this on any of our routers, so can't fix it for sure.

[suzikuo]

I also have related issues. After I joined the local area network, the company's WiFi often couldn't connect and the network was disconnected. Since I shut down/uninstalled it, there have been no issues with the company's WiFi for a month now.

[DarkArc]

I'm seeing this on pfSense 24.03. I suspect this might be related to https://github.com/miniupnp/miniupnp/issues/715.

pfSense moved to FreeBSD 14 last year https://docs.netgate.com/pfsense/en/latest/releases/versions.html while miniupnp seemingly has issues starting with FreeBSD 14.

Are others using BSD based routers/miniupnp based setups?

I'm seeing what (I believe to be) non-ZeroTier PMP activity as well with similar characteristics.

I suspect this isn't a ZeroTier bug at all but an (unfortunately increasingly) widespread issue with this common NAT-PMP and UnPnP implementation.

EDIT: See also https://redmine.pfsense.org/issues/15470