impermanence
impermanence copied to clipboard
How do I know when a file or directory will be mounted during the boot process?
I am moving my configuration to impermanence, using btrfs subvolumes to erase root on each boot. The persistent directory is on another subvolume in the same btrfs partition, and /nix
is inside the persisted subvol
I have hit a number of issues with programs not having the needed files, and did a bit of digging into the system logs.
Given the following impermanence config:
environment.persistence."/persistent" = {
enable = true;
hideMounts = true;
directories = [
{
directory = "/secrets";
mode = "0700";
user = "root";
group = "root";
}
"/etc/ssh"
"/etc/secureboot"
"/var/lib/nixos"
"/var/lib/systemd"
"/nix"
"/var/log"
];
files = [
"/etc/machine-id"
];
};
home.persistence."/persistent/home/phil" = {
directories = [
".ssh"
".emacs.d"
"syncthing"
}
];
files = [
".local/share/fish/fish_history"
];
allowOther = false;
};
The systemd logs show that /nix
, /var/lib/nixos
and /var/log
are mounted very early in the boot sequence, near the end of stage-1-init.
Then /etc/machine-id
is bound after just stage-2-init, while systemd is running but before journalling is started.
Finally, /etc/secureboot
, /var/lib/systemd
, /secrets
and /etc/ssh
are mounted after a number of other services have started up, including local-fs-pre.target
.
The home manager files are mounted even later, and a number of services have started by the time they are mounted.
Programs that were caught out were syncthing
, which started before the syncthing
config directory had been mounted in my home directory, and sshd
, which did start after /etc/ssh
was mounted, but because NixOS had already created and configured /etc/ssh
, the nixos-configured sshd_config
file was hidden by the bind mount.
I was able to fix syncthing by modifying it's systemd service with the following nixos module:
impermanence-syncthing = {config, lib, pkgs, ...}:
{
systemd.services.syncthing.after = [
"multi-user.target"
];
# Default dependencies cause syncthing-init.service to
# set "Before=multi-user.target" which, along with setting
# "After=multi-user.target" for syncthing.service, leads to
# circular dependency.
# Turning off default dependencies, explicit dependencies were
# sufficient.
systemd.services.syncthing-init.unitConfig.DefaultDependencies = false;
};
I find the order directories are mounted during the boot process confusing and unexpected. I want all the persistent directories and files bound or symlinked before system services start trying to read config files. I expected NixOS to write sshd_config
into /etc/ssh
beside the persisted machine keys.
I think this could be resolved with a good understanding of when persistent entries are mounted. Could someone explain the logic to me?