[Feature Request] Allow different stages within the boot sequence for script execution

[jeff-winn]

The problem:

I've been working on this for roughly 3-4 months now in my off time, and something that should be a very simple problem to solve turned into a massive headache. Essentially I need to add a custom static route that is not supported by the Unifi user interface. For example:

ip route add 10.1.0.0/16 dev vti64 proto static scope link src 10.0.0.1

Basically I've got Azure hosted resources that the UDM itself needs to reach out and touch, and it always grabs the WAN IP when opening the connection to the remote server rather than an IP that has access to the S2S VPN subnet. I traced everything going on with my UDM back to the route being missing, so it's a critical path item that the route has to be added.

The interesting thing about this is that depending on WHEN within the unifi-os container startup sequence the route gets added, it does play a role in whether adding a route will actually work.

The fix:

As for what to do about it, I simply modifed the After value in the Unit section of udm-boot.service to network-online.target rather than network.target

Obviously this won't work for everyone (and it might not even be the best option for me), so what I'd propose is adding a second service that runs at a later stage in the boot sequence. The folder that could be targeted within the host OS could be something like on_booted.d rather than on_boot.d to signify the difference of when the execution would take place.

How to replicate:

Have a UDM establish a Site to Site VPN with Microsoft Azure and use "ip route get " via the SSH terminal to see which IP it selects to start the connection on itself. It should use the WAN IP. This could be a conditional DNS forwarder (common problem on the Ubiquiti community site) or anything else that you want to off-load from the UDM.

[boostchicken]

We already do this. the filename determines which is run. A sh file that starts with "00" (e.g. 00-custom_route.sh) will be run first.

Beyond the unifi-os container is started at S95, then udm-boot.service is invoked when the network comes up FOR THE unifi-os container. Did moving to network-online.target fix all your problems? I am considering making that the default.

There is really no better way to add another stage. If you want to test for network connectivity I suggest you do it in your startup script.

Also, 10-dns.sh adds routes just fine and never needed to be moved later. Why does yours?

[jeff-winn]

DNS was fine because of how it interacted with the host OS. It would modify the process and then restart the service ran by the host OS.

With the routes they don’t appear to get added until the container that owns the routes starts up and sets up the routing table (at least that I could see). Anything I added in there was always lost on reboot. If I tested the script it worked, after reboot it wasn’t there.

When I said adding a second stage i meant having two services rather than just the one. If you had a udm-boot-online.service along with the current udm-boot.service that got installed you’d effectively be able to swap out when you need something to run by changing the folder the shell script is sitting in. I’m not sure whether checking the network within my script would fix the timing issue I was seeing, but I’ll give it a go.

Changing the target did fix it, however on reboot sometimes the route would take a while (5-10 minutes before it’d show up).