packages icon indicating copy to clipboard operation
packages copied to clipboard

Published 20 hours ago • verified publisher icon openwrt

miniupnpd-nftables: rules are created but no traffic is being forwarded

Open escape0707 opened this issue 3 years ago • 74 comments

Maintainer

@stintel @ldir-EDB0 @neheb

Environment

  • Model | Xiaomi Redmi Router AC2100
  • Architecture | MediaTek MT7621 ver:1 eco:3
  • Target Platform | ramips/mt7621
  • Firmware Version | OpenWrt SNAPSHOT r18806-2120cad38d / LuCI Master git-22.025.79016-22e2bfb
  • Kernel Version | 5.10.100

Description

As the title says, miniupnpd can't map requested ports successfully for applications and shows "There are no active redirects." in the LuCI web interface.

I setup this environment by:

  1. Flashed the official snapshot version just after I got my hand on this Redmi Router, with the official guide provided method.
  2. Setup 2.4G & 5G WiFi.
  3. Setup PPPoE on WAN.
  4. Installed luci-app-upnp through opkg.

I don't know where to continue the troubleshooting. If any additional information is needed, please let me know. I do have a dynamic global IPv4 address on my router, and I disguised it. If that's needed, please also let me know.

Latest release version 21.02.1 with miniupnpd_2.2.1-3 doesn't have this problem but can't support Xbox / Windows teredo UPnP.

More logs / configs

qbittorrent_download qbittorrent_log.txt logread.txt teredo_log.txt etc_config_upnpd.txt ip_addr_show.txt nftables.txt opkg_info_miniupnpd.txt

escape0707 avatar Feb 15 '22 14:02 escape0707

Redirects are there:

table inet miniupnpd {
	chain forward {
		type filter hook forward priority -25; policy accept;
		iif "pppoe-wan" th dport 49965 @nh,128,32 0xc0a8010b @nh,72,8 0x11 accept
		iif "pppoe-wan" th dport 9564 @nh,128,32 0xc0a8010b @nh,72,8 0x6 accept
		iif "pppoe-wan" th dport 9564 @nh,128,32 0xc0a8010b @nh,72,8 0x11 accept
	}
}
table ip miniupnpd {
	chain prerouting {
		type nat hook prerouting priority dstnat; policy accept;
		iif "pppoe-wan" udp dport 49965 dnat to 192.168.1.11:49965
		iif "pppoe-wan" tcp dport 9564 dnat to 192.168.1.11:9564
		iif "pppoe-wan" udp dport 9564 dnat to 192.168.1.11:9564
	}

	chain postrouting {
		type nat hook postrouting priority srcnat; policy accept;
	}
}
table ip6 miniupnpd {
	chain prerouting {
		type nat hook prerouting priority dstnat; policy accept;
	}

	chain postrouting {
		type nat hook postrouting priority srcnat; policy accept;
	}
}

Please report a LuCI issue instead.

stintel avatar Feb 15 '22 14:02 stintel

Thanks for your quick reply.

Redirects are there

That's what confused me, too. As application's like qbittorrent and windows teredo all say that they can't be connected from outside.

If I manually forward all ports to my testing machine, without the assist of UPnP, then they both report cone or public connectable.

Also, I tested this with canyouseeme.org and the listening port of qbittorrent. UPnP get connection refused, manually forwarding ports works as intended. This is tested on both Windows 10 LTSC 2021 and latest ArchLinux.

escape0707 avatar Feb 15 '22 14:02 escape0707

@stintel Could you suggest me some more accurate ways to test which part is malfunctioning? Thanks!

escape0707 avatar Feb 15 '22 14:02 escape0707

That's what confused me, too. As application's like qbittorrent and windows teredo all says they can't be connected from outside.

Don't trust the application, trust tcpdump. I actually verified miniupnpd-nftables like that recently, because somewhere in some horrible 1500+ message thread in the forum where different issues are discussed, making it impossible to follow anything, and reminding me why I used to stay away from forums, someone complained that it didn't work.

On a remote host:

$ echo foo | nc -u 87.227.x.x 3074

On my OpenWrt router running miniupnpd-nftables:

# tcpdump -ni switch.54 port 3074
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on switch.54, link-type EN10MB (Ethernet), capture size 262144 bytes
16:40:37.990057 IP 94.225.x.x.38515 > 192.168.54.35.3074: UDP, length 4

stintel avatar Feb 15 '22 14:02 stintel

trust tcpdump

I'll go and learn about this tool now! Thanks for the information!

escape0707 avatar Feb 15 '22 14:02 escape0707

If in fact that confirms the port forward doesn't work, please change the title of this issue to reflect just that. The fact that LuCI doesn't show the redirects should be fixed in LuCI so is not relevant for this issue (tracker).

stintel avatar Feb 15 '22 14:02 stintel

The fact that LuCI doesn't show the redirects should be fixed in LuCI so is not relevant for this issue (tracker).

Duly noted, I'll file another issue later on.

escape0707 avatar Feb 15 '22 14:02 escape0707

I used my phone's standalone cell data and in it's terminal run:

$ echo foo | nc -u 111.226.<my public ipv4> 9564

Then on my router, run:

# tcpdump -ni pppoe-wan port 9564
......
15:03:51.885771 IP 106.119.<my phone cell data ipv4>.41998 > 111.226.<my public ipv4>.9564: UDP, length 4
......

At the meanwhile:

# tcpdump -ni br-lan port 9564

doesn't show my phone's ipv4 address, albeit other IP addresses that are currently transferring torrents with my qbittorrent client got captured. And those packets are sent from / to my laptop's LAN ipv4 192.168.1.11

escape0707 avatar Feb 15 '22 15:02 escape0707

My gut feeling says it's related to using ppp. @dangowrt reported a similar problem here.

stintel avatar Feb 15 '22 15:02 stintel

My gut feeling says it's related to using ppp.

Do you think if I "use another router to do the PPPoE dial up, connect my Redmi OpenWRT router to the first one's LAN port and use DHCP to connect to the Internet, then manually forward all ports from the first router to OpenWRT" will help diagnose this problem?

escape0707 avatar Feb 15 '22 15:02 escape0707

I'll have to try that tomorrow, parents are about to sleep. Thank you for helping me trouble shooting, nice sir!

escape0707 avatar Feb 15 '22 15:02 escape0707

Sadly, with this config, I get a lot of:

Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: HTTP REQUEST from [::ffff:192.168.1.11]:33287 : POST /ctl/IPConn (HTTP/1.1)
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: Host: 192.168.1.1:5000
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: SOAPAction: urn:schemas-upnp-org:service:WANIPConnection:2#AddPortMapping
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: AddPortMapping: ext port 10659 to 192.168.1.11:10659 protocol TCP for: qBittorrent/4.5.0alpha1 leaseduration=604800 rhost=
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: UPnP permission rule 0 matched : port mapping accepted
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: Check protocol tcp for port 10659 on ext_if wan 192.168.2.116, 7402A8C0
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: redirecting port 10659 to 192.168.1.11:10659 protocol TCP for: qBittorrent/4.5.0alpha1
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: Returning UPnPError 501: ActionFailed
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: HTTP REQUEST from [::ffff:192.168.1.11]:33957 : POST /ctl/IPConn (HTTP/1.1)
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: Host: 192.168.1.1:5000

full_system_log.txt

The first level router is set to dial up through PPPoE, then set the OpenWrt router as DMZ. When I manually forward all ports also in OpenWrt, I can connect to my machine from the Internet. But when I switch to UPnP, UPnP just won't work, and applications reports so, too.

etc_config_upnpd.txt

ip_addr_show.txt

escape0707 avatar Feb 16 '22 02:02 escape0707

Sadly, with this config, I get a lot of:

Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: HTTP REQUEST from [::ffff:192.168.1.11]:33287 : POST /ctl/IPConn (HTTP/1.1)
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: Host: 192.168.1.1:5000
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: SOAPAction: urn:schemas-upnp-org:service:WANIPConnection:2#AddPortMapping
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: AddPortMapping: ext port 10659 to 192.168.1.11:10659 protocol TCP for: qBittorrent/4.5.0alpha1 leaseduration=604800 rhost=
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: UPnP permission rule 0 matched : port mapping accepted
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: Check protocol tcp for port 10659 on ext_if wan 192.168.2.116, 7402A8C0
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: redirecting port 10659 to 192.168.1.11:10659 protocol TCP for: qBittorrent/4.5.0alpha1
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: Returning UPnPError 501: ActionFailed
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: rule with label 'qBittorrent Enhanced/4.4.0.10' is not a IGD pinhole
Wed Feb 16 02:45:44 2022 daemon.info miniupnpd[6819]: HTTP REQUEST from [::ffff:192.168.1.11]:33957 : POST /ctl/IPConn (HTTP/1.1)
Wed Feb 16 02:45:44 2022 daemon.debug miniupnpd[6819]: Host: 192.168.1.1:5000

The first level router is set to dial up through PPPoE, then set the OpenWrt router as DMZ. When I manually forward all ports also in OpenWrt, I can connect to my machine from the Internet. But when I switch to UPnP, UPnP just won't work, and applications reports so, too.

etc_config_upnpd.txt

ip_addr_show.txt

me too！openwrt pppoe

snakwu avatar Feb 16 '22 03:02 snakwu

@snakwu I think you are facing a completely different problem. You should open an independent issue addressing it.

escape0707 avatar Feb 16 '22 04:02 escape0707

Actually, I don't know what's the meaning of this table:

table inet miniupnpd {
	chain forward {
		type filter hook forward priority -25; policy accept;
		iif "pppoe-wan" th dport 49965 @nh,128,32 0xc0a8010b @nh,72,8 0x11 accept
		iif "pppoe-wan" th dport 9564 @nh,128,32 0xc0a8010b @nh,72,8 0x6 accept
		iif "pppoe-wan" th dport 9564 @nh,128,32 0xc0a8010b @nh,72,8 0x11 accept
	}
}

The default policy in its base chain is ACCEPT, while all rules in it says ACCEPT, too. What does the @nh filters verifies? This table currently does nothing because so far there isn't anything malicious that needed to be rejected, yet?

escape0707 avatar Feb 16 '22 04:02 escape0707

I manually specified my external_ip in /etc/config/upnpd and UPnP set up the firewall rules successfully now. But I'm still getting the similar behavior that tcpdump shows incoming testing packets only on wan not br-lan. Manually forwarding still works.

escape0707 avatar Feb 16 '22 05:02 escape0707

I think I know where the problem is. If I add any port forward rule manually and UPnP seems to work. But I still need to do more tests.

escape0707 avatar Feb 16 '22 08:02 escape0707

When at least one port forward is defined, certain additional rules are enabled by firewall4, like the rule that automatically accepts all inbound traffic that is related to a DNAT'ed connection

jow- avatar Feb 16 '22 08:02 jow-

Yes, I diffed those nft rulesets and saw several ct status dnat accept. That is the reason why only enabling miniupnpd is not working.

escape0707 avatar Feb 16 '22 08:02 escape0707

After set up UPnP and add any firewall rules through LuCI, everything works just fine, for both DHCP and PPPoE.

The display issue of luci-app-upnp is already reported here https://github.com/openwrt/luci/issues/5678

escape0707 avatar Feb 16 '22 14:02 escape0707

What do you mean specifically with "add any firewall rules through LuCI"? I am looking at miniupnpd atm and try to figure out how to improve it

jow- avatar Feb 16 '22 16:02 jow-

@jow- I mean, when I only installed luci-app-upnp and enabled UPnP, then launch an app (qbittorrent) to make UPnP port forward request, I will get a nftables rule set like this: nftables_without_manually_port_forwarding.txt

At this time, the port forward rules made by miniupnpd won't work. I can now add a non-related port's forward rule manually from LuCI, and this time I get the following nftables rule set: nftables_with_manually_port_forwarding.txt

A diff between those two shows that two

ct status dnat accept

rules were inserted just before

jump reject_from_wan

and

jump reject_to_wan

which I believe are what make the port forwarded by miniupnpd works again.

escape0707 avatar Feb 17 '22 06:02 escape0707

@stintel I'm not using PPPoE, got IPoE FTTH with the ISP-provided router operating in bridge-mode and OpenWrt device connected to it receiving a public IPv4 via DHCP. Yet rules added by miniupnpd didn't have any effect.

dangowrt avatar Feb 17 '22 12:02 dangowrt

Same here, but my issue is with PCP and ipv6. I see miniupnpd responding to the port open request, and I see the rules in nftables yet they do nothing.

Also, the lease file stays empty (I believe this is another separate issue)

tiagogaspar8 avatar Feb 17 '22 12:02 tiagogaspar8

@jow already explained the reason here I have static port forwards so I have that rule and that's why things work for me.

stintel avatar Feb 17 '22 12:02 stintel

I'm sorry, I'm confused, do we need to add something then?

I also saw that miniupnpd v2.3.0 fixes some issues with nftables and might be related to this, no?

tiagogaspar8 avatar Feb 17 '22 22:02 tiagogaspar8

@tiagogaspar8 https://github.com/openwrt/packages/issues/17871#issuecomment-1042615077

I think those rules added by luci firewall setting is the key point.

escape0707 avatar Feb 18 '22 01:02 escape0707

Yes, but what I mean is that I bleive the new release of miniupnpd fixes this issue because it corrects the location and the jumps that are currently broken.

Also, the port forwarding rules don't explain the reason why ipv6 PCP doesn't work.

tiagogaspar8 avatar Feb 18 '22 01:02 tiagogaspar8

Oh, I don't have IPv6 currently. And i guess i should update my OpenWrt and miniupnpd to test it again.

escape0707 avatar Feb 18 '22 01:02 escape0707

@escape0707 Yeah, I wanted to update my miniupnpd version too to see if this issue is fixed magically (hoping)

If you'd like you can try ipv6 with a he.net free tunnel 😁

tiagogaspar8 avatar Feb 18 '22 01:02 tiagogaspar8