plugins icon indicating copy to clipboard operation
plugins copied to clipboard

25.7.8 update killed LAN-to-LAN connections

Open jeffrpowell opened this issue 1 month ago • 3 comments

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

  • [x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
  • [x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

I was doing active work on my LAN around 12:00-12:30 UTC today (26 Nov 2025) when suddenly all LAN-to-LAN access was broken. I am unable to connect (ICMP ping, HTTP) to my gateway IP, nor to any other IP on my LAN. Just getting connection timeouts.

Connecting to the internet / WAN is still functional. Interestingly, Tailscale SSH to my LAN machines is still functional (except for OPNSense, of course).

The post time of the 25.7.8 release on https://forum.opnsense.org/index.php?topic=49869.msg253323#msg253323 is 12:17 UTC 26 Nov 2025. It's gotta be a problem with an automatic update.

I did perform the obligatory box restart. There was a small blip where I could access the admin page over HTTP, but pretty quickly it went away.

Apologies, it'll be hard to get into the box to retrieve any logs or anything.

I'm running OPNSense in a VM on Proxmox 8.1. Gigabyte X570 I AORUS PRO WIFI Mini ITX AM4 motherboard.

jeffrpowell avatar Nov 26 '25 13:11 jeffrpowell

Maybe related but my tailscale is now not working with 25.7.8. Shows no peers. Seeing the below in Backend logs:

'2025-11-26T13:19:32-05:00 Error configd.py [18111006-9e50-44d2-b00e-42bc878f2b9a] returned exit status 1 2025-11-26T13:19:22-05:00 Error configd.py [971fc6af-1c26-4c84-83f7-b6c32e65d2ed] returned exit status 1 2025-11-26T13:19:17-05:00 Error configd.py [e3d0804d-f0c0-4e28-8694-9eb9f60fd151] returned exit status 1 2025-11-26T13:19:12-05:00 Error configd.py [a402a7a2-d6d5-4b11-a10a-d8ec5ecc1e05] returned exit status 1 2025-11-26T13:15:23-05:00 Error configd.py [88d8174d-2cf9-4965-aa23-7e51982834b3] returned exit status 1 2025-11-26T13:07:54-05:00 Error configd.py [171862c9-4784-4f07-b631-a72c190ac7ab] Script action stderr returned "b'2025/11/26 13:07:54 No DERP map from tailscaled; using default.\n2025/11/26 13:07:54 attempting to fetch a DERPMap from https://controlplane.tailscale.com\n2025/11/26 13:07:54 portmap: monitor: gateway and self IP changed: gw=50.174.187.1 self=50.104.118.8'" 2025-11-26T13:07:51-05:00 Error configd.py [d7b36504-123c-4f0f-a568-1e4ae79f6f51] returned exit status 1 2025-11-26T13:06:48-05:00 Error configd.py [6f30cae4-ca3c-43d7-acf7-821eea9bb0aa] returned exit status 1 2025-11-26T13:06:40-05:00 Error configd.py [9abf7535-ad5a-452b-832e-a67f82e2ea82] Script action stderr returned "b'Failed to connect to local Tailscale daemon for /localapi/v0/status; not running? Error: dial unix /var/run/tailscale/tailscaled.sock: connect: no such file or directory'" 2025-11-26T13:06:38-05:00 Error configd.py [988ba8fd-9d4b-40fc-8459-efe13ad2c347] Script action stderr returned "b'Failed to connect to local Tailscale daemon for /localapi/v0/status; not running? Error: dial unix /var/run/tailscale/tailscaled.sock: connect: no such file or directory'" 2025-11-26T13:06:28-05:00 Error configd.py [9c211865-1267-49ff-852b-a1b7badf5ab9] Script action stderr returned "b'2025/11/26 13:06:28 No DERP map from tailscaled; using default.\n2025/11/26 13:06:28 attempting to fetch a DERPMap from https://controlplane.tailscale.com\n2025/11/26 13:06:28 portmap: monitor: gateway and self IP changed: gw=50.174.187.1 self=50.174.188.8'" 2025-11-26T13:05:32-05:00 Error configd.py [c8437b9e-44be-4059-96b4-6b8ccdbdaf37] Script action stderr returned "b'2025/11/26 13:05:32 No DERP map from tailscaled; using default.\n2025/11/26 13:05:32 attempting to fetch a DERPMap from https://controlplane.tailscale.com\n2025/11/26 13:05:32 portmap: monitor: gateway and self IP changed: gw=50.174.187.1 self=50.174.188.8'" 2025-11-26T13:05:31-05:00 Error configd.py [cd7f763f-5096-4519-b3cb-c52dcf764d1e] returned exit status 1'

arraylabs avatar Nov 26 '25 18:11 arraylabs

This might be a Tailscale problem after all. I disabled Tailscale on my client machine and I was suddenly able to access local IPs again. As it turns out, my instance of OPNSense never upgraded to 25.7.8 (I was on 25.7.6). Sure enough, Tailscale also recently updated on 25 November 2025 (https://tailscale.com/changelog#2025-11-25). I'm starting to shift my focus towards that camp.

jeffrpowell avatar Nov 28 '25 05:11 jeffrpowell

It seems I chose a bad configuration value in the os-tailscale OPNSense plugin. I set all my local IPs as advertised subnet routes. See https://tailscale.com/kb/1023/troubleshooting#lan-traffic-prioritization-with-overlapping-subnet-routes for recommendations. My short-term fix was to wipe out that setting + layer on the defense by applying --accept-routes=false on all my servers with Tailscale installed.

jeffrpowell avatar Nov 28 '25 18:11 jeffrpowell