serokell
Password-based ssh authentication does not work
I think this may fall into the general category of "deploy-rs leaves the terminal in a broken state", and is probably related to #105 and #78 and https://github.com/serokell/deploy-rs/issues/49#issuecomment-766187567.
I was trying out deploy-rs by deploying to a vm, and to keep things simple was deploying as root, with ssh authentication being by password. I get asked for the ssh (root@localhost) Password: twice: for the copy and for the activation, but never (seem to) get a password request for the rollback (I think - what each ssh connection is for is rather unclear to me). However, this is just a broken interaction with the terminal (xterm) - as seen when using SSH_ASKPASS_REQUIRE=prefer, see below.
The deploy seems to hang at 👀 ℹ️ [wait] [INFO] Waiting for confirmation event... until it times out. I see:
$ deploy --ssh-opts='-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p 2221'
🚀 ℹ️ [deploy] [INFO] Running checks for flake in .
warning: unknown flake output 'deploy'
🚀 ℹ️ [deploy] [INFO] Evaluating flake in .
🚀 ℹ️ [deploy] [INFO] The following profiles are going to be deployed:
[mySystem.system]
user = "root"
ssh_user = "root"
path = "/nix/store/6d9ialdrnps4hc5b1rclqq6v1gp30aaw-activatable-nixos-system-payload-21.05.20210712.3386dc3"
hostname = "localhost"
ssh_opts = ["-o", "UserKnownHostsFile=/dev/null", "-o", "StrictHostKeyChecking=no", "-p", "2221"]
🚀 ℹ️ [deploy] [INFO] Building profile `system` for node `mySystem`
🚀 ℹ️ [deploy] [INFO] Copying profile `system` to node `mySystem`
Warning: Permanently added '[localhost]:2221' (ED25519) to the list of known hosts.
(root@localhost) Password:
🚀 ℹ️ [deploy] [INFO] Activating profile `system` for node `mySystem`
🚀 ℹ️ [deploy] [INFO] Creating activation waiter
Warning: Permanently added '[localhost]:2221' (ED25519) to the list of known hosts.
Warning: Permanently added '[localhost]:2221' (ED25519) to the list of known hosts.
(root@localhost) Password:
👀 ℹ️ [wait] [INFO] Waiting for confirmation event...
then a long pause, until finally
👀 ❌ [wait] [ERROR] Error waiting for activation: Timeout elapsed for confirmation
🚀 ❌ [deploy] [ERROR] Waiting over SSH resulted in a bad exit code: Some(1)
🚀 ℹ️ [deploy] [INFO] Revoking previous deploys
However, this is just a terminal problem: if I mash keys enough during that long pause, I can trigger an actual password prompt:
$ deploy --ssh-opts='-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p 2221' -s
🚀 ℹ️ [deploy] [INFO] Evaluating flake in .
🚀 ℹ️ [deploy] [INFO] The following profiles are going to be deployed:
[mySystem.system]
user = "root"
ssh_user = "root"
path = "/nix/store/6d9ialdrnps4hc5b1rclqq6v1gp30aaw-activatable-nixos-system-payload-21.05.20210712.3386dc3"
hostname = "localhost"
ssh_opts = ["-o", "UserKnownHostsFile=/dev/null", "-o", "StrictHostKeyChecking=no", "-p", "2221"]
🚀 ℹ️ [deploy] [INFO] Building profile `system` for node `mySystem`
🚀 ℹ️ [deploy] [INFO] Copying profile `system` to node `mySystem`
Warning: Permanently added '[localhost]:2221' (ED25519) to the list of known hosts.
(root@localhost) Password:
🚀 ℹ️ [deploy] [INFO] Activating profile `system` for node `mySystem`
🚀 ℹ️ [deploy] [INFO] Creating activation waiter
Warning: Permanently added '[localhost]:2221' (ED25519) to the list of known hosts.
Warning: Permanently added '[localhost]:2221' (ED25519) to the list of known hosts.
(root@localhost) Password:
⭐ ℹ️ [activate] [INFO] Activating profile
stopping the following units: -.mount, nix-.ro\x2dstore.mount, nix-.rw\x2dstore.mount, nscd.service, sshd.service, systemd-udevd-control.socket, systemd-udevd-kernel.socket, systemd-udevd.service, tmp-shared.mount, tmp-xchg.mount
Failed to stop -.mount: Job type stop is not applicable for unit -.mount.
activating the configuration...
removing user ‘sshd’
setting up /etc...
removing obsolete symlink ‘/etc/ssh/moduli’...
removing obsolete symlink ‘/etc/ssh/sshd_config’...
removing obsolete symlink ‘/etc/pam.d/sshd’...
reloading user units for root...
setting up tmpfiles
reloading the following units: dbus.service, firewall.service
restarting the following units: dhcpcd.service
starting the following units: -.mount, nscd.service, systemd-udevd-control.socket, systemd-udevd-kernel.socket
the following new units were started: httpd.service, logrotate.timer, systemd-timesyncd.service, time-sync.target
⭐ ℹ️ [activate] [INFO] Activation succeeded!
⭐ ℹ️ [activate] [INFO] Magic rollback is enabled, setting up confirmation hook...
⭐ ℹ️ [activate] [INFO] Waiting for confirmation event...
asdf
(root@localhost) Password:
ControlSocket /home/ben/.ssh/root@localhost:2221 already exists, disabling multiplexing
🚀 ℹ️ [deploy] [INFO] Success activating, attempting to confirm activation
🚀 ℹ️ [deploy] [INFO] Deployment confirmed.
Notice how there is also some weirdness: sometimes (second example) it actually activates the profile, and sometimes it does not (first example). If it does not, I can mash the keyboard in similar way to get a password prompt, but this prompt does not work well: I need to hit enter twice to submit a password, and often it is rejected, though I have once managed to get it accepted after a few attempts:
👀 ℹ️ [wait] [INFO] Waiting for confirmation event...
asdf
(root@localhost) Password:
(root@localhost) Password:
root@localhost's password:
Permission denied, please try again.
root@localhost's password:
⭐ ℹ️ [activate] [INFO] Activating profile
(Notice the change in prompt). It feels like the keyboard is not working properly in that terminal.
Unlike #105, the terminal seems fine after deploy-rs exits.
If I set SSH_ASKPASS_REQUIRE=prefer, so ssh will spawn a x11 window to ask for the password, then things work a lot better and the deployment succeds as expected.
(There is a bit of weirdness: we seem to spawn two windows at the same time, and they fight over grabbing the keyboard, but this resolves itself and is only a minor annoyance. I suspect this is related to why (with terminal-based password prompt) it sometimes activates and sometimes not -- presumably two ssh connections are raced, and the result depends on if the "activation" one comes first and gets the terminal password input or not?)
(There is a bit of weirdness: we seem to spawn two windows at the same time, and they fight over grabbing the keyboard, but this resolves itself and is only a minor annoyance. I suspect this is related to why (with terminal-based password prompt) it sometimes activates and sometimes not -- presumably two ssh connections are raced, and the result depends on if the "activation" one comes first and gets the terminal password input or not?)
This can be avoided by using a different askpass program, by setting SSH_ASKPASS. The obvious packages from nixos are x11_ssh_askpass, ssh-askpass-fullscreen, lxqt.lxqt-openssh-askpass and ksshaskpass.
x11_ssh_askpass is the "default" and the only one having the above problem. This seems to be the only one that tries to grab the keyboard. All the others pop up multiple windows as expected which don't clash with each other.
nix copy does not show the password promt at all, but waits silently for input, if you input the password, it will exit with 0. checked via:
strace nix copy --no-check-sigs --to ssh://[email protected] /nix/store/nysd6q63iw2lpb44fqm6c5k2xrvb9nff-activatable-home-manager-generation
