microvm.nix
microvm.nix copied to clipboard
astro
Allow unlimited restart attempts for autostart VMs
Currently when the VMs initialize during system startup, they may fail because some bridges created by systemd-networkd may be not available yet:
Jun 29 13:44:30 vm systemd[1]: Starting MicroVM 'microvm-test'...
Jun 29 13:44:30 vm systemd[1]: Started MicroVM 'microvm-test'.
Jun 29 13:44:30 vm microvm@microvm-test[11347]: /run/wrappers/bin/qemu-bridge-helper: cannot run `/run/libvirt/nix-helpers/qemu-bridge-helper': No such file or directory
Jun 29 13:44:30 vm microvm@microvm-test[11339]: qemu-system-x86_64: -netdev bridge,id=eth0,br=br-untrusted,helper=/run/wrappers/bin/qemu-bridge-helper: bridge helper failed
Jun 29 13:44:30 vm systemd[1]: [email protected]: Main process exited, code=exited, status=1/FAILURE
Jun 29 13:44:30 vm systemd[1]: [email protected]: Failed with result 'exit-code'.
Jun 29 13:44:31 vm systemd[1]: [email protected]: Scheduled restart job, restart counter is at 5.
Jun 29 13:44:32 vm systemd[1]: Stopped MicroVM 'microvm-test'.
Jun 29 13:44:32 vm systemd[1]: Dependency failed for MicroVM 'microvm-test'.
Jun 29 13:44:32 vm systemd[1]: [email protected]: Job [email protected]/start failed with result 'dependency'.
Jun 29 16:25:23 vm systemd[1]: Starting MicroVM 'microvm-test'...
Jun 29 16:25:23 vm systemd[1]: Started MicroVM 'microvm-test'.
When I retry to launch the service manually in about 30s it launches perfectly.
It would be good to have an option for limitless boot retry attempts.
The [email protected] already includes Restart=always. Is that not sufficient?
You may also extend systemd.services."microvm@".after to solve the actual cause here though automatic restarts should work.
The [email protected] already includes Restart=always. Is that not sufficient?
Seems like it's not, for some reason it makes 3 attempts then stops.
You may also extend systemd.services."microvm@".after to solve the actual cause here though automatic restarts should work.
Could you please give me an idea how? I'm not a nix expert yet :)
I believe that's the explanation why just "restart=always" is not enough: https://unix.stackexchange.com/questions/289629/systemd-restart-always-is-not-honored
Maybe it's a good idea to allow custom properties in the generated unit files?
The second answer in that link explains that it is just rate limit what you are seeing. systemd is going to continue starting/restarting all over again after a while.
For some reason that "after a while" never happens. I've set up 3 vms, one failed to start and never re-attempted to start in a couple of weeks. Okay, sometimes systemd is a bit weird.