Add option to prevent unmounting and exporting ZFS pool

[TECHNOFAB11]

I just tried to use nixos-anywhere for a new server and noticed it now unmounts and exports the ZFS pool. I need these to stay mounted though so that I can copy over the SSH keys for SOPS. An option to prevent that would be nice, like no-reboot, maybe no-u(n)mount?

[sedlund]

I have the same use case. It's actually in disko here:

https://github.com/nix-community/disko/blob/d07de570ba05cec2807d058daaa044f6955720c7/lib/default.nix#L440

haven't dug in yet to see how it could be bypassed.

[TECHNOFAB11]

In the nixos-anywhere.sh at the end it also exports all ZFS pool, I don't think the Disko destroy script is currently used for that as that presumably deletes partitions?

[sedlund]

yeah i think I jumped the gun before I had to head out. I looked at that script and it tests for zpool command and does that recurive unmount.

I'm on btrfs though so I dont think that's my issue.

I decided I'm going to pre-provision fresh ssh keys and put them in sops before running nixos-anywhere with --extra-files to copy them onto the host like this: https://github.com/nix-community/nixos-anywhere/blob/4c94cecf3dd551adf1359fb06aa926330f44e5a6/docs/howtos/secrets.md#example-decrypting-an-openssh-host-key-with-pass

then the initial build will work and I won't have to come back after to update my secrets, rebuild, and redeploy.

[Mic92]

Yes --extra-files should be preferably used to bootstrap secrets. Would be cool if someone could make this more clear in our quickstart if this is not already the case - Maybe even some sops-nix section...

[fin444]

--extra-files should be preferably used to bootstrap secrets

Isn't it insecure to create secrets on one machine and copy them to another?

[sedlund]

Isn't it insecure to create secrets on one machine and copy them to another?

Not if done correctly. If secrets are clear text and written to files, they should be written correctly permissioned to tmpfs (ram), or encrypted storage, network copied over a secure transport (ssh), and stored similarly on the remote.

This isnt automated yet, and cloud-init is overwriting the keys I copy with --extra-files after reboot. But maybe useful for ideas. Excuse the mess.

[Mic92]

--extra-files should be preferably used to bootstrap secrets

Isn't it insecure to create secrets on one machine and copy them to another?

With sops-nix, your local user can decrypt your servers secrets already, so also having the decryption key, doesn't change a lot.

[Mic92]

Isn't it insecure to create secrets on one machine and copy them to another?

Not if done correctly. If secrets are clear text and written to files, they should be written correctly permissioned to tmpfs (ram), or encrypted storage, network copied over a secure transport (ssh), and stored similarly on the remote.

This isnt automated yet, and cloud-init is overwriting the keys I copy with --extra-files after reboot. But maybe useful for ideas. Excuse the mess.

My personal preference these days is to generate just the age key for use in sops and have the ssh key just be a secret like everything else. Than you can rotate both independently and also deploy secrets to machines without an ssh service running. In clan we have then a facts module generating the actual ssh key for each host automatically: https://git.clan.lol/clan/clan-core/src/commit/0f95bfd279b12865382f0ffd3459086090217fa1/clanModules/sshd/default.nix#L22

and store the results back into sops-nix automatically.

In clan we also wired up update and install commands in a way that they automatically provision secrets. For nixos-anywhere, we also use the --extra-files argument: https://git.clan.lol/clan/clan-core/src/commit/0f95bfd279b12865382f0ffd3459086090217fa1/pkgs/clan-cli/clan_cli/machines/install.py#L60

This implementation also uses a temporary directory as mentioned by @sedlund

[JeanMertz]

I have a use-case in which I have a persistent directory (with contents) that must be present on the host before the reboot happens. The extra-files argument is insufficient as my backup of that persistent directory is done in such a way that permissions and ownership are preserved in the archive, so I need to manually go into the host, and extract the archive in the correct position to keep all the relevant ownership/permission flags. That's not possible with --extra-files.

Preventing unmounting would make this workflow possible/easy, by extracting the archive into /mnt/persist and then triggering a reboot.