openwrt
pbr: No default routes for IPv6
Maintainer: @stangri (find it by checking history of the package Makefile) Environment: (put here arch, model, OpenWrt version) X86 in a proxmox vm, openwrt 22.03.5, pbr version: 1.1.1-7 using nft
Description:
I'm running a wireguard client and server in openwrt, and I'm using pbr to only route my trusted network (192.168.2.0/24, fd00::1ced:cafe::20/64) through the VPN client, everything else can go through WAN.
So I added a policy to route 192.168.2.0/24 fc00:1ced:cafe::20/64 through vpn interface in the prerouting chain, and the ipv6 support is enabled. After this, I was able to get VPN working for ipv4 (tested on my dev machine in trusted zone), however, I couldn't access any ipv6 sites, and ping6 google.com hangs forever:
$ ping6 google.com
PING6(56=40+8+8 bytes) fd00:1ced:cafe:20:<RANDOM_SUFFIX> --> 2607:f8b0:4006:80b::200e
Then I ran /etc/init.d/pbr status on the openwrt host and saw
IPv6 table 257 route: unreachable default dev lo metric 1024 pref medium
So it seems that an ipv6 default route to the VPN client is not properly setup. I must admit that I don't have too much knowledge of ipv6 routing in general, but I'm more than happy to hack the script for further debugging if given a direction!
More background information:
-
My VPN provider unfortunately only hands out /128 ipv6 ULA, so I had to run NAT66 (masquerade for ipv6 on the vpn client zone in firewall settings). Maybe that makes ipv6 routing a little bit more complex. Moreover, I had to disable ipv6 delegation from my ISP (ipv6 prefix filter set to
localon trusted interface) and disable source routing on wan6 interface as well. -
Just for sake of debugging, I removed pbr and used the default route (set route_allowed_ips in the wireguard client) for the VPN client and ipv6 works as expected, so the VPN client setup should be OK.
Here's the entire output from /etc/init.d/pbr status, hope that helps!
Redacted information: <WAN_IPV4>: public ipv4 from ISP <WAN_IPV6>: public ipv6 (/64) from ISP <VPN_IPV4>: VPN exit node public ipv4 <VPN_IPV6>: VPN exit node public ipv6 <VPN_ASSIGNED_ULA>: the ULA (/128) given by VPN provider <FE80_LINK_LOCAL>: link-local address
root@OpenWrt:~# /etc/init.d/pbr status
============================================================
pbr - environment
pbr 1.1.1-7 running on OpenWrt 22.03.5. WAN (IPv4): wan/eth0/<WAN_IPV4>. WAN (IPv6): wan6/eth0/<WAN_IPV6>.
============================================================
Dnsmasq version 2.86 Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
============================================================
pbr chains - policies
chain pbr_forward { # handle 57
}
chain pbr_input { # handle 58
}
chain pbr_output { # handle 59
}
chain pbr_prerouting { # handle 60
ip saddr @pbr_vpn_4_src_ip_cfg036ff5 goto pbr_mark_0x020000 comment "all" # handle 10256
ip6 saddr @pbr_vpn_6_src_ip_cfg036ff5 goto pbr_mark_0x020000 comment "all" # handle 10257
}
chain pbr_postrouting { # handle 61
}
============================================================
pbr chains - marking
chain pbr_mark_0x010000 { # handle 10244
counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 10245
return # handle 10246
}
chain pbr_mark_0x020000 { # handle 10247
counter packets 2526 bytes 316824 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 10248
return # handle 10249
}
============================================================
pbr nft sets
set pbr_vpn_4_src_ip_cfg036ff5 { # handle 10254
type ipv4_addr
flags interval
auto-merge
comment "all"
elements = { 192.168.2.0/24 }
}
set pbr_vpn_6_src_ip_cfg036ff5 { # handle 10255
type ipv6_addr
flags interval
auto-merge
comment "all"
elements = { fd00:1ced:cafe:20::/64 }
}
============================================================
IPv4 table 256 route: default via <WAN_IPV4> dev eth0
IPv4 table 256 rule(s):
30000: from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv6 table 256 route: default via <FE80_LINK_LOCAL> dev eth0 proto static metric 512 pref medium
unreachable default dev lo metric 1024 pref medium
IPv6 table 256 rule(s):
<WAN_IPV6> dev eth0 proto static metric 1024 pref medium
<VPN_IPV6> via <FE80_LINK_LOCAL> dev eth0 proto static metric 512 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via <FE80_LINK_LOCAL> dev eth0 proto static metric 512 pref medium
unreachable default dev lo metric 1024 pref medium
IPv4 table 257 route: default via <VPN_IPV4> dev mullvadnyc1
IPv4 table 257 rule(s):
30001: from all fwmark 0x20000/0xff0000 lookup pbr_mullvadnyc1
IPv6 table 257 route: unreachable default dev lo metric 1024 pref medium
IPv6 table 257 rule(s):
<VPN_ASSIGNED_ULA> dev mullvadnyc1 proto kernel metric 256 pref medium
unreachable default dev lo metric 1024 pref medium
You can add an ipv6 route by yourself in /etc/config/network Something like:
config route6
option interface 'wgc0'
option target '::/0'
option metric '256'
option table 'pbr_wgc0'
Adapt interface and table to your config. Make sure to set option masq6 '1' in your wg/vpn zone config. Make sure forwarding is allowed from your lan to wg/vpn zone. Setup your pbr rules as usual and it should work for both ipv4 and ipv6. No need to disable prefix delegation. pbr setups the proper rules but no default routing table, so you have to setup it by yourself as described above.
@schm0 Thanks a lot! that indeed fixed the issue!
Keeping this issue open for pbr to add default routing table
You can add an ipv6 route by yourself in /etc/config/network Something like:
config route6 option interface 'wgc0' option target '::/0' option metric '256' option table 'pbr_wgc0'Adapt interface and table to your config. Make sure to set option masq6 '1' in your wg/vpn zone config. Make sure forwarding is allowed from your lan to wg/vpn zone. Setup your pbr rules as usual and it should work for both ipv4 and ipv6. No need to disable prefix delegation. pbr setups the proper rules but no default routing table, so you have to setup it by yourself as described above.
@schm0 is it something you believe pbr should be adding to the tables it creates?
Without a proper route this obviously can't work. PBR copies the routes already in place. For WAN/WAN6 this is no problem because there is usually already a "catchall" default route in place (0.0.0.0, ::/0) But for tunnel connections in combination with policy based routing this is most likely not the case because it would route all traffic through the tunnel and that defeats the purpose of policy based routing.
I don't know... PBR creates it own tables anyway.... Why not use 0.0.0.0, ::/0 for those?
The question is... why does the ipv4 pbr_wgc0 table has a 0.0.0.0 target but the ipv6 one has no ::/0 target?
Without a proper route this obviously can't work. PBR copies the routes already in place. For WAN/WAN6 this is no problem because there is usually already a "catchall" default route in place (0.0.0.0, ::/0) But for tunnel connections in combination with policy based routing this is most likely not the case because it would route all traffic through the tunnel and that defeats the purpose of policy based routing.
I don't know... PBR creates it own tables anyway.... Why not use 0.0.0.0, ::/0 for those?
The question is... why does the ipv4 pbr_wgc0 table has a 0.0.0.0 target but the ipv6 one has no ::/0 target?
Would something like this work then? Nevermind the Makefile changes, just the extra line in the init script?
Can you explain what your intention with this change is?
Going through all that code is a bit much.
|| ipv4_error=1 this should be ipv6_error=1 ?
I'm trying to figure out why the pbr tables for IPv4 do have a 0.0.0.0 target but the IPv6 don't have a ::/0 target. Did pbr copy the IPv4 route from the wan interface? Or where does this come from?
Isn't it just easier to let pbr create tables with 0.0.0.0 and ::/0 ? What is the actual reason to copy the routes over to the pbr route tables?
Bumping this issue.
In the meantime, creating a static route sort of works. Except, when reloading pbr I have to disable then re-enable the rule to get ipv6 connectivity again.
Fixed in 1.1.7 you can grab a binary (both ipk and apk) from the upstream repos.
Thanks! I can confirm it works flawlessly without the static ipv6 route in Version 1.1.7-59!