plugins icon indicating copy to clipboard operation
plugins copied to clipboard

Published 20 hours ago • verified publisher icon opnsense

os-wireguard 2.4_1 | missing route to/from to virtual IPs of type Proxy ARP | after upgrading OPNsense to 23.7.7

Open relume opened this issue 2 years ago • 35 comments

Important notices Before you add a new report, we ask you kindly to acknowledge the following:

  • [X] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
  • [X] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
  • [X] The title contains the plugin to which this issue belongs

Describe the bug

  • After upgrading OPNsense from version 23.7.5 to OPNsense to 23.7.6 and 23.7.7 (23.7.7_1/2/3) routing to/from Wireguard interfaces/networks to virtual IPs of type Proxy ARP (public IPs 1:1 to internal IPs) do not work any more. It worked on OPNsense 23.7.5 and before, even after reboots. This is the case for all configured Wireguard interfaces/networks (1 mobile/road warrior, 2 site to site). Routing to defined LAN networks do work.

  • Restarting wireguard by WebGUI or cli does not enable routing again.

  • We configured for testing a single route (System/Routes/Configuration) entry and adding the routing configuration form one virtual IP Proxy ARP (mypublicIP/32) and as gateway one automatically defined wireguard gateway. By applying this routing configuration, routing to/from all defined wireguard interfaces/networks come up to all defined Proxy ARP virtual IPs (not only the single Proxy ARP virtual IP just defined.

  • But routing routing to/from all defined wireguard interfaces/networks to defined Proxy ARP virtual IPs does not survive any rebooting. Only reapplying the routing configuration brings up the routing again. Thus I am not shure if the issue is concerning the os-wireguard plugin or more a general routing/gatway problem.

To Reproduce

  • at the moment only configuring the above described routing configuration form one virtual IP Proxy ARP (mypublicIP/32) and as gateway one automatically defined wireguard gateway and reapplying it after reboot is reproducible on our configuration.

Expected behavior

  • routing routing to/from Wireguard interfaces/networks to virtual IPs of type Proxy ARP should work/come up at every start/restart of OPNsense as it was on version 23.7.5.

Screenshots

Relevant log files

  • wireguard log : Error | wireguard | /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: not a valid interface gateway address: '
  • general log : Error | opnsense | /usr/local/etc/rc.routing_configure: ROUTING: not a valid interface gateway address: ''
  • routing log : no concerning errors

Additional context

Environment

OPNsense 23.7.7_3-amd64 FreeBSD 13.2-RELEASE-p3 OpenSSL 1.1.1w 11 Sep 2023 CPU type | QEMU Virtual CPU version 2.5+ (6 cores, 6 threads)

relume avatar Nov 06 '23 16:11 relume

Thank you for creating an issue. Since the ticket doesn't seem to be using one of our templates, we're marking this issue as low priority until further notice.

For more information about the policies for this repository, please read https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md for further details.

The easiest option to gain traction is to close this ticket and open a new one using one of our templates.

OPNsense-bot avatar Nov 06 '23 16:11 OPNsense-bot

Might work with the latest changes for 23.7.8 but first time I’m hearing this use case so not sure.

fichtner avatar Nov 06 '23 16:11 fichtner

Thank you. 23.7.8 is actually not yet available on the "normal" firmware update channel. So I can not test.

Just to clarify why our public addresses defined as virtual IP Proxy ARP should be also accessible over wireguard VPN (when connected by wireguard) and not directly over the public route: only few services are enabled to be public, other services as ssh etc. are only accessible within LAN and to our defined wireguard VPNs. Thus those single public IPs are declared in the wireguard peer configurations as "AllowedIPs".

relume avatar Nov 06 '23 16:11 relume

It should be https://github.com/opnsense/plugins/commit/af80514ad

feel free to test:

# opnsense-patch -c plugins af80514ad

fichtner avatar Nov 06 '23 18:11 fichtner

I applied the patch af80514ad and rebooted.

Unfortunately the patch has no effect to the described issue. Again only reapplying (entered into edit mode then clicking ok without any changes) the route entry mentioned above brought up the correct routing.

best

relume avatar Nov 07 '23 08:11 relume

It might be better to share the full settings (replacing private data)

AdSchellevis avatar Nov 07 '23 08:11 AdSchellevis

Thanks. Which parts/aspects of the settings are of concrete interest ( as screenshots from WebGUI or as snippets from the actual config file)?

relume avatar Nov 07 '23 09:11 relume

snippets from the config for the relevant parts are likely best.

AdSchellevis avatar Nov 07 '23 09:11 AdSchellevis

hoping that obfuscation of private data did not harm clearness. many thanks. opnsense_2.7.7_3_config_snippets_20231107.xml.zip

relume avatar Nov 07 '23 10:11 relume

I don't think this has anything todo with wireguard to be honest, your wan is xxx.yyy.zzz.242/29, the next hop is xxx.yyy.zzz.241 according to the gateways, but has a "far gateway" property set and then fails at:

https://github.com/opnsense/core/blob/b787a35c8e05bdff3df9fcbf098a6e5f8964a3e2/src/etc/inc/system.inc#L556-L559

Which is odd for two reasons, the file supplied has a "gateway" set and likely doesn't need a host route at all. The proxyarp entries live on the wan interface, which appears to be a vtnet, also not really related to Wireguard.

AdSchellevis avatar Nov 07 '23 12:11 AdSchellevis

Thanks for your reponse and sorry to "bother".

  • In front of our OPNsense Firewall we have an other OPNsense only routing instance with the IP xxx.yyy.zzz.241/29 for our public subnet xxx.yyy.zzz.240/29 on the same vtnet as our OPNsense Firewall - that is correct.

  • When I configurated our OPNsense Firewall instance (with WAN-IP xxx.yyy.zzz.242/29) first in august 2023 (version from 2023.07.31), the only way to get the Proxy ARP virtual IPs to work, was to set for the upstream default Gateway the option "far gateway" (even it did not convince me).

  • I disabled now this setting (option "far gateway") and accessibility/routing to the Proxy ARP virtual IPs came down immediately from wiregurad networks. A reboot did not bring up again routing to/from Proxy ARP virtual IPs. Before toggling the "fat gateway" option every reboot (on Version 23.7.5) routing to/from Proxy ARP virtual IPs was brougth up again for either point of origin: public, LANs and for wireguard networks (as reported).

  • Reenabling the "fat gateway" option and rebooting does not bring up routing to/from Proxy ARP virtual IPs and the wireguard networks.

  • Actually/now the only possibility to bring up routing to/from Proxy ARP virtual IPs and the wireguard again after a reboot is to enter in the WebGUI in the default upstream gateway settings and reapplying it without any changes. Or the same procedure (again withou any changes) with the "dummy" single routing entry to one of the Proxy ARP virtual IPs (mentioned above) even if this routing entry is actually disabled.

  • Honestly my OPNsense proficiency is to "fractional" to understand why those manual WebGUI interactions that should have no effect as there are no setting changes bring up the routing to/from Proxy ARP virtual IPs and the wireguard networks, but a reboot does not (after the updates from 23.7.5 to 23.7.6 and 23.7.7...).

  • I would appreciate any hint how to solve and/or to narrow down this issue. And it seems as you wrote not to be related to wireguard dirctly but affects routing to/from wireguard networks for the defined Proxy ARP virtual IPs.

relume avatar Nov 08 '23 15:11 relume

This lies somewhat beyond community support scope for me, the use of proxy arp entries is not very common to be honest, when these proxy entries actually lie within the same network as the primary interface (as the config suggests) a normal virtual ip might be more suitable.

The forum (https://forum.opnsense.org) might be a better place to discuss network design choices.

AdSchellevis avatar Nov 08 '23 16:11 AdSchellevis

Thank you - I can understand your point about the support question. I thought it coud be a kind of bug as it worked up to 23.7.5 flawless until the following udates to 23.7.6 etc. By the way, my first configuration approach in august was with a "normal" virtual IP and I was not able to get it to work at all. In the forum I was told to use proxy arp virtual IPs as the only way to make work this type of configuration and it worked immediately by using proxy arp virtual IPs. best regards

relume avatar Nov 08 '23 16:11 relume

which forum topic was that? just interested if I see something obvious in relation to your issue

AdSchellevis avatar Nov 08 '23 16:11 AdSchellevis

it was on the forum post Firewall | 1:1 / One-to-One NAT single IPs for multiple public single IPs where I asked about the 1:1 mapping.

relume avatar Nov 08 '23 17:11 relume

well, if xxx.yyy.zzz.244 lies inside the wan network, an alias is more logical in my humble opinion. After adding an alias, I would try to ping from that source address using the firewall to some external address to validate if the address works. If that works, move on to the nat rules.

AdSchellevis avatar Nov 08 '23 18:11 AdSchellevis

thanks again.

  1. I gave it a try with alias virtual IP for an additional public IP xxx.yyy.zzz.246 with an NAT One-to-One mapping to an LAN IP as the other settings mentioned previously .
  2. A route was inserted automatically for this public IP xxx.yyy.zzz.246 and ping from wireguard networks started immediately to work (setting this additional public IP beforhand in the wireguard client/peer Allowed IPs setting.). Thus I changed the other proxy arp virtual IPs also to alias virtual IPs. Automatic created routes were inserted an ping from wireguard networks worked now also for those public IPs. But tcp access from wireguard networks to those public IPs xxx.yyy.zzz.244 - 246 did not work, even as I opened for testing the incoming Firewall rules fully.
  3. I swichted from Manual outbound NAT rule generation to Automatic outbound NAT rule generation but no automatic rules were generated. Normal LAN to Internet connection went down, but tcp access from wireguard networks to the public IPs xxx.yyy.zzz.244 - 246 started. I switched back to Manual outbound NAT rule generation and tcp access from wireguard networks still worked.
  4. After rebooting OPNsense tcp access from wireguard networks to the public IPs (244 - 246) was down again. I tried to repeat this effect from point 3. only by oppening (toggling) the default NAT outbound rule for the WAN interface with Interface-address as NAT-address and saving the rule without any change. TCP access from wireguard networks to the public IPs (244 - 246) started immediately.
  5. Updated to OPNsense version 23.7.8 with reboot and repeated point 4. to get TCP access from wireguard networks.
  6. As no additional or changed autogenerated firewall rules were shown/after "toggling" the default NAT outbound rule I had a look at the cli level with the command pfctl -sa to show possible differences direct after reboot and before togglin the default NAT outbound rule and also after having toggled the default NAT rule.
  7. The differences before/after toggling are obvious:

by "toggling" (after reboot) OPNsense inserted the following additional NAT rules (those rules do not survive a reboot):

10.92.1.10 - 12 : internalt IPs mapped One-To-One to public IPs (alias) xxx.yyy.zzz.244/32 - 246/32

TRANSLATION RULES:
no nat proto carp all
...
nat on wg2 inet from (wg2:network) to 10.92.1.10-> (wg2) port 1024:65535 round-robin
nat on wg1 inet from (wg1:network) to 10.92.1.10 -> (wg1) port 1024:65535 round-robin
nat on wg3 inet from (wg3:network) to 10.92.1.10 -> (wg3) port 1024:65535 round-robin
...
nat on wg2 inet from (wg2:network) to 10.92.1.11-> (wg2) port 1024:65535 round-robin
nat on wg1 inet from (wg1:network) to 10.92.1.11 -> (wg1) port 1024:65535 round-robin
nat on wg3 inet from (wg3:network) to 10.92.1.11 -> (wg3) port 1024:65535 round-robin
...
etc.
...
rdr on wg2 inet from any to xxx.yyy.zzz.244 -> 10.92.1.10 bitmask
rdr on wg1 inet from any to xxx.yyy.zzz.244 -> 10.92.1.10 bitmask
rdr on wg3 inet from any to xxx.yyy.zzz.244 -> 10.92.1.10 bitmask
...
etc.
...
  1. Thus I am wondering, why "toggling" the default NAT outbound rule, which has nothing to do with the One-to-One mapping and the wireguard networks, has the effect that the missing rules for the wireguard networks are inserted by OPNsense but not by default at startup and thus do not survive a reboot?

relume avatar Nov 11 '23 10:11 relume

The missing outbound nat might be a bug introduced by https://github.com/opnsense/core/commit/6ea9d216e2a717c594f2a95568bda7366dfea4d2 , we're looking into alternatives as this is likely a side affect and the reload shouldn't be needed at that point.

AdSchellevis avatar Nov 11 '23 14:11 AdSchellevis

Thank you for reporting.

relume avatar Nov 14 '23 10:11 relume

Updated today to 23.7.8_1-amd64 with os-wireguard 2.5_1. Above nat and rdr rules for wg interfaces survived the installation of the update but not the following reboot.

relume avatar Nov 15 '23 21:11 relume

I swichted from Manual outbound NAT rule generation to Automatic outbound NAT rule generation but no automatic rules were generated.

This is expected. WireGuard VPN doesn't offer automatic rule generation because it doesn't have an (assigned) interface IPv4 configuration and that's the same for all tunnel types.

As far as #6852 goes this pertains to rc.linkdown, but shouldn't affect bootup. If after reboot this doesn't work try running:

# /usr/local/etc/rc.filter_configure

to see if that fixes your issue. The thing I'm wondering, however is why it's concluding there is no address on WireGuard interface, because that comes from the early configuration through WireGuard options. Can you provide the full wireguard log and the system log for "ifconfig" errors after bootup and probable fix through filter configure script?

fichtner avatar Nov 16 '23 08:11 fichtner

PS: What is your "Tunnel address" for that particular WireGuard instance?

fichtner avatar Nov 16 '23 08:11 fichtner

Thank you.

This is expected. WireGuard VPN doesn't offer automatic rule generation because it doesn't have an (assigned) interface IPv4 configuration and that's the same for all tunnel types.

I intended all types of automatic rule generation such standard outbound rules from LAN to WAN. But hopped also Wireguard VPN outbound rules would be generated automatically. Good to know.

# /usr/local/etc/rc.filter_configure

Executing manually above command brings up/adds the missing nat and rdr rules etc. after a reboot.

wireguard log | 2023.11.16 | no entries before reboot, log contains only entries after reboot (2023-11-16T16:35:49)
xxx.yyy.zzz.242 = fw.mydomain.net

<37>1 2023-11-16T16:36:11+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="1"] wireguard instance wg_mobile (wg1) can not reconfigure without stopping it first.
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="2"] wireguard instance wg_mobile (wg1) stopped
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="3"] wireguard instance wg_mobile (wg1) started
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="4"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt2'
<35>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="5"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: not a valid interface gateway address: ''
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="6"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (,WG_MOBILE_GW)
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="7"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (execute task : dpinger_configure_do(,WG_MOBILE_GW))
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="8"] wireguard instance wg_hb_campora (wg2) can not reconfigure without stopping it first.
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="9"] wireguard instance wg_hb_campora (wg2) stopped
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="10"] wireguard instance wg_hb_campora (wg2) started
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="11"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt4'
<35>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="12"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: not a valid interface gateway address: ''
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="13"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (,WG_CAMPORA_GW)
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="14"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (execute task : dpinger_configure_do(,WG_CAMPORA_GW))
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="15"] wireguard instance wg_prosana (wg3) can not reconfigure without stopping it first.
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="16"] wireguard instance wg_prosana (wg3) stopped
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="17"] wireguard instance wg_prosana (wg3) started
<37>1 2023-11-16T16:36:12+01:00 fw.mydomain.net wireguard 31443 - [meta sequenceId="18"] /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt3'

I could not find any entry for "ifconfig" in the system log neither messages for/related to executing manually the command rc.filter_configure after login (at 2023-11-16T16:37:10) as admin by ssh :

system log | 2023.11.16 | no entries before reboot, log contains only entries after reboot (2023-11-16T16:35:49)

xxx.yyy.zzz.242 = fw.mydomain.net

<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="18"] <118> wg_campora (wg2) -> v4: 10.242.81.80/32
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="19"] <118> wg_mobile (wg1) -> v4: 10.242.10.1/25
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="20"] <118> wg_prosana (wg3) -> v4: 10.242.70.80/32
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="21"] <118>
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="22"] <118> HTTPS: SHA256 xxxx
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="23"] <118>               xxxx
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="24"] <118> SSH:   SHA256 xxxx (ECDSA)
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="25"] <118> SSH:   SHA256 xxxx (ED25519)
<13>1 2023-11-16T16:36:18+01:00 fw.mydomain.net kernel - - [meta sequenceId="26"] <118> SSH:   SHA256 xxxx (RSA)
<85>1 2023-11-16T16:37:10+01:00 fw.mydomain.net sudo 47138 - [meta sequenceId="27"]    admin : TTY=pts/0 ; PWD=/home/admin ; USER=root ; COMMAND=/usr/bin/su

actual boot log:

2023-11-16T16:36:12+01:00 dpinger_configure_do[31443] done.
2023-11-16T16:36:12+01:00 system_routing_configure[31443] Setting up routes for opt3...
2023-11-16T16:36:12+01:00 system_routing_configure[31443] done.
2023-11-16T16:36:12+01:00 wireguard_configure_do[269] done.
2023-11-16T16:36:12+01:00 ntpd_configure_do[269] Starting NTP service...
2023-11-16T16:36:13+01:00 ntpd_configure_do[269] done.
2023-11-16T16:36:13+01:00 unbound_configure_do[269] Starting Unbound DNS...
2023-11-16T16:36:14+01:00 unbound_configure_do[269] done.
2023-11-16T16:36:14+01:00 rrd_configure[269] Generating RRD graphs...
2023-11-16T16:36:14+01:00 rrd_configure[269] done.

wireguard interfaces tunnel addresses (wg1, wg2, wg3: all wg intances have the same problem):

wg1 | wg_mobile | 10.242.10.1/25, client peers 10.242.10.10 - 10.242.10.126/32
wg2 | wg_campora | 10.242.81.80/32, client network peer 10.242.81.81/32, remote network 10.81.0.1/16
wg3 | wg_prosana | 10.242.70.80/32, client network peer 10.242.70.70/32, remote network 10.70.0.1/16

As far I can see there is no log entry that shows the difference before and after manually executing the command rc.filter_configure. May I enable some additional log option?

relume avatar Nov 16 '23 16:11 relume

I had a look also at the configd log. I seems that at boot up OPNsense/Filter generated is generated before WG instances init. But after WG init OPNsense/Unbound/core is recalled again. After manual execution of command rc.filter_configure (at 2023-11-16T16:39:08) only OPNsense/Filter is regenerated.

configd log | 2023.11.16 | reboot (2023-11-16T16:35:49)

xxx.yyy.zzz.242 = fw.mydomain.net

<13>1 2023-11-16T16:35:10+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="38257"] [2b98bad0-58d7-443f-94b4-68f9f288f002] Reboot system
<13>1 2023-11-16T16:36:07+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="1"] [e11bf910-6faf-4abe-82b5-15a30b572867] request pf current overall table record count and table-entries limit
<13>1 2023-11-16T16:36:07+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="2"] [3a4173ed-2b5a-42b9-b755-0a9009477af7] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="3"] [6a554a56-2c03-4090-8dca-4947fa4dbd5f] Linkup starting vtnet1
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="4"] [f6601ff7-c7a0-4aa8-b194-f2151ad2b69b] Linkup starting vtnet2
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="5"] [f6a85012-ebfc-42b0-b67c-d9418e0021c7] generate template OPNsense/Filter
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="6"] generate template container OPNsense/Filter
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="7"] [a7ddc1c2-a265-41fd-b7fa-d08369846657] Linkup starting vtnet0
<15>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="8"]  OPNsense/Filter generated //usr/local/etc/filter_tables.conf
<15>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="9"]  OPNsense/Filter generated //usr/local/etc/filter_geoip.conf
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="10"] [2d5263f2-2a03-49b2-acfb-0ecf2d67c664] refresh url table aliases
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="11"] [7049b0fa-ad09-459c-9a5d-e3de065aa1e4] Unbound cache flush
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="12"] [8cfd7659-50af-4462-8ad7-73552cf44e89] generate template OPNsense/WebGui
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="13"] generate template container OPNsense/WebGui
<15>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="14"]  OPNsense/WebGui generated //usr/local/etc/php.ini
<15>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="15"]  OPNsense/WebGui generated //usr/local/lib/php.ini
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="16"] [e65e4a09-9ae0-4bd9-90f4-502952a53b30] show system routing table
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="17"] [acef4191-eee0-4632-85b1-c77644964758] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="18"] [4ed8fb74-fa4a-4733-a772-30659235104b] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="19"] [4087c5a4-59f9-4ce8-bf0c-c97edc17f47c] generate template OPNsense/Unbound/*
<13>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="20"] generate template container OPNsense/Unbound/core
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="21"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/access_lists.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="22"]  OPNsense/Unbound/* generated //var/unbound/advanced.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="23"]  OPNsense/Unbound/* generated //usr/local/etc/unbound/unbound-blocklists.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="24"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/safesearch.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="25"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/dot.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="26"]  OPNsense/Unbound/* generated //var/unbound/private_domains.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="27"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/domainoverrides.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="28"]  OPNsense/Unbound/* generated //var/unbound/root.hints
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="29"]  OPNsense/Unbound/* generated //usr/local/etc/unbound_dhcpd.conf
<15>1 2023-11-16T16:36:09+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="30"]  OPNsense/Unbound/* generated //var/unbound/dnsbl_module.py
<13>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="31"] [cddd9e9e-98f3-48dc-b99a-649de27e709e] generate template OPNsense/Filter
<13>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="32"] generate template container OPNsense/Filter
<15>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="33"]  OPNsense/Filter generated //usr/local/etc/filter_tables.conf
<15>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="34"]  OPNsense/Filter generated //usr/local/etc/filter_geoip.conf
<13>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="35"] [0e9697d9-6148-466b-81eb-626aa7cb0f1e] refresh url table aliases
<14>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="36"] message 0e9697d9-6148-466b-81eb-626aa7cb0f1e [] returned   
<13>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="37"] [a0361de1-cb44-43fa-bf57-d6413e34c639] IPsec list legacy VirtualTunnelInterfaces
<14>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="38"] message 2d5263f2-2a03-49b2-acfb-0ecf2d67c664 [] returned {"status": "ok"}   
<13>1 2023-11-16T16:36:11+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="39"] [2c7a036e-940d-487f-9dd9-9c11840cf46e] configure wireguard instances ()
<13>1 2023-11-16T16:36:12+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="40"] [0495c88c-16bb-4a84-89f9-5f7353de3e9d] show system routing table
<13>1 2023-11-16T16:36:12+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="41"] [18dc7bca-deba-4124-97ad-71b3dc091eb9] show system routing table
<13>1 2023-11-16T16:36:12+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="42"] [642cc27d-dd07-4b42-9c0a-36cf7ed59991] show system routing table
<13>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="43"] [023f26ff-cb4d-4fe3-a265-e25d36044bf5] Unbound cache dump
<13>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="44"] [e0dcd01c-21c7-4b8a-8e63-77135d220381] generate template OPNsense/Unbound/*
<13>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="45"] generate template container OPNsense/Unbound/core
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="46"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/access_lists.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="47"]  OPNsense/Unbound/* generated //var/unbound/advanced.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="48"]  OPNsense/Unbound/* generated //usr/local/etc/unbound/unbound-blocklists.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="49"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/safesearch.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="50"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/dot.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="51"]  OPNsense/Unbound/* generated //var/unbound/private_domains.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="52"]  OPNsense/Unbound/* generated //usr/local/etc/unbound.opnsense.d/domainoverrides.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="53"]  OPNsense/Unbound/* generated //var/unbound/root.hints
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="54"]  OPNsense/Unbound/* generated //usr/local/etc/unbound_dhcpd.conf
<15>1 2023-11-16T16:36:13+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="55"]  OPNsense/Unbound/* generated //var/unbound/dnsbl_module.py
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="56"] [22a64f46-7276-480d-a0a3-226f50ef0dae] Restarting syslog
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="57"] [20f7c2b6-4db6-445c-a9ed-a77a4ba055c2] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="58"] [b3fa2865-8d7c-4e63-86e9-dab1407cfdf9] restarting cron
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="59"] [5fa58af9-ca2c-484f-80b2-84eb0d73ec37] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="60"] [3cbf6242-bf5f-423e-a19d-0de95c412820] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="61"] [e49bfd97-a950-4d7a-884b-71bde3c2e3e5] generate template OPNsense/Syslog
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="62"] generate template container OPNsense/Syslog
<13>1 2023-11-16T16:36:15+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="63"] [175dbdde-2dce-41a5-879a-01b6443852f4] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="64"] [dcb28898-aff2-40d1-8734-ac9bad8db9dc] generate template OPNsense/Cron
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="65"]  OPNsense/Syslog generated //etc/rc.conf.d/syslog_ng
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="66"]  OPNsense/Syslog generated //etc/newsyslog.conf
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="67"]  OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="68"]  OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/syslog-ng-destinations.conf
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="69"]  OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/syslog-ng-local.conf
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="70"]  OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/syslog-ng-lockout.conf
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="71"]  OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/syslog-ng-config-events.conf
<13>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="72"] generate template container OPNsense/Cron
<15>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="73"]  OPNsense/Cron generated //var/cron/tabs/nobody
<13>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="74"] [36a5ed01-502d-4b13-874a-70e41e6dd0f6] List syslog applications
<13>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="75"] [53c7e945-8f3e-436e-8047-8ba36220894e] Archive syslog files
<13>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="1"] [b8e96b0b-2f88-4223-b6cb-346260d8f282] configure openvpn instances
<14>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="2"] message 53c7e945-8f3e-436e-8047-8ba36220894e [] returned OK  
<13>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="3"] [98e3d671-e526-416b-9577-c2706ceaf40f] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="4"] [50bed38e-505f-46b6-81b7-638bdf1ea993] IPsec list legacy VirtualTunnelInterfaces
<14>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="5"] message b8e96b0b-2f88-4223-b6cb-346260d8f282 [] returned OK  
<14>1 2023-11-16T16:36:16+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="6"] message 22a64f46-7276-480d-a0a3-226f50ef0dae [] returned OK  
<13>1 2023-11-16T16:36:30+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="7"] [35e8d9c1-5c5f-4a5d-b777-81f33a4dd2cb] Fetching service list ( )
<13>1 2023-11-16T16:36:30+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="8"] [7ac5a76d-d0c5-4d06-bc41-63c16970c587] request traffic stats
<13>1 2023-11-16T16:36:30+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="9"] [76f8cea2-de30-4929-baed-090283099dc3] list gateway status
<13>1 2023-11-16T16:36:30+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="10"] [58c4f987-a278-4123-8de9-8f9e2b1ac58a] Retrieve firmware product info
<13>1 2023-11-16T16:36:30+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="11"] [aa1a8340-2d8b-46a1-816e-9eadfc8499ca] request traffic stats
<13>1 2023-11-16T16:36:30+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="12"] [3d4ce535-4f22-46d4-aa06-28ef03d77ca7] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:36:30+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="13"] [558a1a7e-b497-44db-a83f-6ae40005d5c5] system status
<13>1 2023-11-16T16:36:35+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="14"] [51e191ae-58df-42ca-8459-e80b8c2df3b0] list gateway status
<13>1 2023-11-16T16:36:35+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="15"] [f48b9d02-fbee-47a8-b0f1-d036100b3596] request traffic stats
<13>1 2023-11-16T16:36:35+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="16"] [e63f9d73-5c8b-4fd4-8eb3-228ade223fe6] Fetching service list ( )

...

<13>1 2023-11-16T16:39:02+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="154"] [443eef50-271f-48bc-9242-0edc12049058] IPsec list legacy VirtualTunnelInterfaces
<13>1 2023-11-16T16:39:03+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="155"] [933c7ef8-30cf-4430-b068-749481a74f04] Retrieve firmware product info
<13>1 2023-11-16T16:39:05+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="156"] [0397d3d8-2bd3-48ac-93b2-69f41fcf7920] request traffic stats
<13>1 2023-11-16T16:39:07+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="157"] [a4600f27-e91d-435e-91e1-a61695121613] list gateway status
<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="158"] [515ac427-f97e-40ed-8161-b0121dae9383] request pf current overall table record count and 

should be the moment when executing manually command "manually executing the command rc.filter_configure"

<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="158"] [515ac427-f97e-40ed-8161-b0121dae9383] request pf current overall table record count and table-entries limit
<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="159"] [db4ebdf7-e2f6-4da7-beea-d0d6f917dde5] generate template OPNsense/Filter
<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="160"] generate template container OPNsense/Filter
<15>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="161"]  OPNsense/Filter generated //usr/local/etc/filter_tables.conf
<15>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="162"]  OPNsense/Filter generated //usr/local/etc/filter_geoip.conf
<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="163"] [adff69f7-b48f-47a8-bfde-bde7e58f4805] refresh url table aliases
<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="164"] [64b3d0d9-684f-4e9e-a280-595ce4e0f70b] Fetching service list ( )
<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="165"] [f4a5a7a3-4a9b-46b3-805c-fcb2a13896d9] Retrieve firmware product info
<13>1 2023-11-16T16:39:08+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="166"] [15d624e4-d0c9-4774-8d4f-47ba0d55a12d] IPsec list legacy VirtualTunnelInterfaces
<14>1 2023-11-16T16:39:10+01:00 fw.mydomain.net configd.py 236 - [meta sequenceId="167"] message adff69f7-b48f-47a8-bfde-bde7e58f4805 [] returned {"status": "ok"}

relume avatar Nov 16 '23 16:11 relume

Sorry for so many question. This came up in our meeting today and I want to specifically ask:

  1. Do you have an interface assigned to each wireguard instance?
  2. If yes on 1. do you have a gateway created as well?
  3. If ye son 2. do you have the interface IPv4 mode set to static?
  4. Does this actually only happen with the automatic NAT for WAN or do you have manual NAT rules that do not work.

The background here is that interface IPv4 mode setting was never intended for Wireguard and also doesn't work anymore on the latest versions. The real spot for setting WireGuard addresses is "tunnel address" under the respective WireGuard instance settings. The automatic nat rules were byproduct of this wrong IPv4 interface setting, too.

Cheers, Franco

fichtner avatar Nov 20 '23 13:11 fichtner

  1. Do you have an interface assigned to each wireguard instance?

wg1 for mobile (road warrior) wg seems to have been created automatically. In Interfacs:Assignments the wg1 entry can not be deleted. wg2 and wg3 for Site-to-Site wg VPN can be deleted, but I cannot remember if I assigned them manually. I gues not. For all wg interface the option "Dynamic gateway policy" is enabled, thus "This interface does not require an intermediate system to act as a gateway" is enabled.

  1. If yes on 1. do you have a gateway created as well?

Gateways (System:Gateway:Single) were created automatically, also for all wg interfaces.

  1. If ye son 2. do you have the interface IPv4 mode set to static?

For all wg interfaces the option "IPv4 Configuration Type" (interfaces:wg...) is set to "None" as by default.

  1. Does this actually only happen with the automatic NAT for WAN or do you have manual NAT rules that do not work

All NAT rules are working (only wg NAT rules do not work after reboot as discussed/reported previously).

  • We have no "NAT: Port Forward" rules.
  • We have 3 manual "NAT: Onte-to-One" rules from public subnet addresses (xxx.yyy.zzz.244/32, 245/32, 246/32) to the corresponding 3 internal LAN IPs.
  • We have 1 manual "NAT: Outbound" rule as Source "any" on "WAN" interface, Destination "*" with "NAT address: WAN address". "Automatic outbound NAT rule generation" does not generate any rule at all. (for wg interfaces they would not be generated anyway, as you mentioned earylier).

The background here is that interface IPv4 mode setting was never intended for Wireguard and also doesn't work anymore on the latest versions. The real spot for setting WireGuard addresses is "tunnel address" under the respective WireGuard instance settings. The automatic nat rules were byproduct of this wrong IPv4 interface setting, too.

I have no option to select/set tunnel address in the IPv4 Configuration Type option list nor elsewhere.

best

relume avatar Nov 20 '23 14:11 relume

Updated today to OPNsense 23.7.9-amd64.

The issue still persists after reboot and I have to run # /usr/local/etc/rc.filter_configure manually in order that nat and rdr rules for wg interfaces are created to get access from wg networks to internal IPs mapped One-To-One to virtual public IPs (alias).

best

relume avatar Nov 26 '23 10:11 relume

So I worked on a root cause today and if you could test the following patch that would be helpful: https://github.com/opnsense/core/commit/64e0867a4

# opnsense-patch 64e0867a4

I'm not sure this will immediately fix it but it would be good to exclude this exact case for your missing rules problem.

fichtner avatar Nov 28 '23 15:11 fichtner

wg1 for mobile (road warrior) wg seems to have been created automatically. In Interfacs:Assignments the wg1 entry can not be deleted.

This isn't automatic. You simply locked the interface (which is ok). Same for the other assigned interfaces. The process is not automatic.

I have no option to select/set tunnel address in the IPv4 Configuration Type option list nor elsewhere.

That option is under WireGuard Instances "Tunnel Address" :)

Cheers, Franco

fichtner avatar Nov 29 '23 08:11 fichtner

Many thanks! I can report the following:

patch 64e0867a4:

  1. rebooted > no nat, rdr rules for wg interfaces to virtual public IPs (alias) as expected.
  2. Installed the patch 64e0867a4 > after patch installation same situation as on 1.
  3. rebooted > same situatioin as on 1.

wg interfaces

That option is under WireGuard Instances "Tunnel Address"

I am sorry I misunderstood your statement about the wg interfaces and IPv4 interfaces:

The background here is that interface IPv4 mode setting was never intended for Wireguard and also doesn't work anymore on the latest versions. The real spot for setting WireGuard addresses is "tunnel address" under the respective WireGuard instance settings. The automatic nat rules were byproduct of this wrong IPv4 interface setting, too.

I looked for an explicit option "tunnel address" in the interface config and wg instance forms. As I have set right from the start the appropriate working tunnel address in CIDR notation I did not expect that you may assume this field. :-)

But nevertheless I can not interpret what are the meaning/consequences of

The automatic nat rules were byproduct of this wrong IPv4 interface setting, too.

So to avoid some "strange" interferences from the already existings wg interfaces I did the following test:

  1. created a new wg instance (wg4) and enabled new wg instance (tunnel address 10.242.11.1/24, no overlapping port, no peers), applied wg setting, checked with # pfctl -sa > no nat, rdr rules for wg4 and all others wg interfaces, neither wg1-3 clients can access virtual public IPs.
  2. created/assigned in WebGUI interfaces new wg4 interface as proposed from the WebGUI, leaving all defaults for the new created interface (not enabled), saved the form, applied the changes, checked with # pfctl -sa > nat, rdr rules for wg1-3 were created at this point, wg1-3 clients can access virtual public IPs.
  3. enabled wg4 interface, saved, checked with # pfctl -sa > now also nat, rdr rules for wg4 to all definied virtual public IPs (alias) are present.
  4. rebooted, checked with # pfctl -sa > no nat, rdr rules for all wg interfaces are present.
  5. instead of running manually # /usr/local/etc/rc.filter_configure after reboot, just disabled wg4 interface, saved and applied changes in the form, checked with # pfctl -sa > now nat, rdr rules for wg1-3 are present (rules for wg4 are not present as expected). Thus it does not matter if the command # /usr/local/etc/rc.filter_configure is run manually or some changes on the WebGUI interface are applied (assuming that later calls /usr/local/etc/rc.filter_configure in some deppending circumstances).

best regards

relume avatar Nov 29 '23 16:11 relume