Clarify expected behaviour when connecting to Vagrant boxes - interaction required?

[phlummox]

Describe the bug

When used from the command line, Hashicorp's vagrant is capable of ssh-ing to a newly created box without any password or user interaction being required to make the connection:

$ vagrant ssh -- echo "hello world"
hello world

But trying to do the same with pyinfra results in a No host key for [127.0.0.1]:2200 found in known_hosts, do you want to continue [y/n] prompt being shown. Is this the expected behaviour? That would seem to make pyinfra strictly less capable than vagrant.

To Reproduce

Tested on Ubuntu 20.04, using Python 3.8.10 and pyinfra 2.3 (installed with pip install pyinfra).

1. Create a Vagrantfile:

Vagrantfile contents

Vagrant.configure("2") do |config|
    config.vm.box = "generic/alpine310"

    config.vm.provider "virtualbox" do |vb|
      vb.memory = "300"
    end
end

2. Bring up the Vagrant box

$ vagrant up

3. Confirm prompt-less ssh works using vagrant ssh

$ vagrant ssh -- echo "hello world"
hello world

4. Try the same with pyinfra:

$ pyinfra --debug @vagrant exec -- echo "hello world"

Expected behavior

The command echo "hello world" should execute on the Vagrant box without any need for user interaction.

Actual behaviour

The user is required to accept a new host key:

$ pyinfra  @vagrant exec -- echo "hello world"
--> Loading config...
--> Loading inventory...
    Getting Vagrant config...

--> Connecting to hosts...
No host key for [127.0.0.1]:2200 found in known_hosts, do you want to continue [y/n] 

Comments

Running pyinfra with --debug suggests it should be using the SSH options "-o UserKnownHostsFile=/dev/null" and "-o StrictHostKeyChecking=no", but for some reason it isn't doing so.

$ pyinfra --debug  @vagrant exec -- echo "hello world"
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    Getting Vagrant config...
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Loading SSH config for default
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Got Vagrant SSH info: 
['Host default', 'HostName 127.0.0.1', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']
    [pyinfra_cli.inventory] Checking possible group_data directory: /mnt/data/dev/vouch-demo
    [pyinfra.connectors.vagrant] Got Vagrant SSH info: 
['Host default', 'HostName 127.0.0.1', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']

--> Connecting to hosts...
    [pyinfra.connectors.ssh] Connecting to: @vagrant/default ({'allow_agent': False, 'look_for_keys': False, 'hostname': '127.0.0.1', '_pyinfra_ssh_forward_agent': None, '_pyinfra_ssh_config_file': None, '_pyinfra_ssh_known_hosts_file': None, '_pyinfra_ssh_strict_host_key_checking': None, '_pyinfra_ssh_paramiko_connect_kwargs': None, 'username': 'vagrant', 'port': 2200, 'timeout': 10, 'pkey': <paramiko.rsakey.RSAKey object at 0x7ffa365e0d00>})
    [pyinfra.connectors.sshuserclient.client] Loading SSH config: None
No host key for [127.0.0.1]:2200 found in known_hosts, do you want to continue [y/n] ^CExiting upon user request!

Meta

output of pyinfra --support

--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Linux
      Platform: Linux-5.13.0-48-generic-x86_64-with-glibc2.29
      Release: 5.13.0-48-generic
      Machine: x86_64
    pyinfra: v2.3
    Executable: /mnt/data/dev/vouch-demo//env/bin/pyinfra
    Python: 3.8.10 (CPython, GCC 9.4.0)

How was pyinfra installed (source/pip)?

Using pip install pyinfra.

output with -vv and --debug

--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    Getting Vagrant config...
localhost: >>> vagrant status --machine-readable
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Loading SSH config for default
localhost: >>> vagrant ssh-config default
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Got Vagrant SSH info: 
['Host default', 'HostName 127.0.0.1', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']
    [pyinfra_cli.inventory] Checking possible group_data directory: /mnt/data/dev/vouch-demo
    [pyinfra.connectors.vagrant] Got Vagrant SSH info: 
['Host default', 'HostName 127.0.0.1', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']

--> Connecting to hosts...
    [pyinfra.connectors.ssh] Connecting to: @vagrant/default ({'allow_agent': False, 'look_for_keys': False, 'hostname': '127.0.0.1', '_pyinfra_ssh_forward_agent': None, '_pyinfra_ssh_config_file': None, '_pyinfra_ssh_known_hosts_file': None, '_pyinfra_ssh_strict_host_key_checking': None, '_pyinfra_ssh_paramiko_connect_kwargs': None, 'username': 'vagrant', 'port': 2200, 'timeout': 10, 'pkey': <paramiko.rsakey.RSAKey object at 0x7f49ed67a8b0>})
    [pyinfra.connectors.sshuserclient.client] Loading SSH config: None
No host key for [127.0.0.1]:2200 found in known_hosts, do you want to continue [y/n]

[phlummox]

Some additional info on this bug/question, in case it's of use to anyone else encountering this issue.

Workaround

A workaround is to explicitly invoke the pyinfra command with the command-line arguments pyinfra --data ssh_known_hosts_file=/dev/null --data ssh_strict_host_key_checking=no.

(But pyinfra's behaviour is still odd, since those same command-line arguments are effectively contained in the output pyinfra gets from vagrant ssh-config, in pyinfra.connectors.vagrant._get_vagrant_ssh_config().)

 

Cause and possible fix (with patch)

When running against a Vagrant box, pyinfra has all the information it needs to make an ssh connection during execution of the function pyinfra.connectors.vagrant.make_names_data():

https://github.com/Fizzadar/pyinfra/blob/0ee6b9f509e9fdc335a23f3de27e5fa7a458a61c/pyinfra/connectors/vagrant.py#L132

_get_vagrant_ssh_config() fetches the ssh config for a Vagrant box by running vagrant ssh-config, and the config is returned as a list of lines, something like

['Host default', 'HostName 127.0.0.1',
'User vagrant', 'Port 2222', 
'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no',
'PasswordAuthentication no',
'IdentityFile /path/to/project_dir/.vagrant/machines/default/virtualbox/private_key',
'IdentitiesOnly yes', 'LogLevel FATAL', ''
]

For each host, all of these keys and values are put into a dictionary and passed to pyinfra.connectors.vagrant._make_name_data(), which "translates" from the key names SSH uses (like "Port", "User", and "IdentityFile") to the ones pyinfra uses (like "ssh_port", "ssh_user", and "ssh_key").

But the two SSH keys relevant to this bug (UserKnownHostsFile = /dev/null, and StrictHostKeyChecking = no) never get so translated. So by the time code in the SSH connector gets executed (e.g. pyinfra.connectors.ssh._make_paramiko_kwargs), it tries to connect with whatever SSH's default values are. (~/.ssh/known_hosts and yes, I believe.)

The fix appears to be to add those two keys to the list of keys that get translated - see the patch here: https://github.com/Fizzadar/pyinfra/compare/2.x...phlummox-dev:pyinfra:fix-vagrant-connector

However, there are currently no end-to-end Vagrant tests for testing this. It should be possible to write some – GitHub's MacOS 10.15 virtual environment includes Vagrant and VirtualBox – but I'm not really familiar enough with pyinfra's codebase to create them.

[Fizzadar]

Thank you for the detailed write up @phlummox! Indeed this is a regression with the recent changes to make SSH host keys behave like OpenSSH. Would you mind PR-ing that fix and I'll get that merged in?

Added #875 to setup e2e vagrant tests 👍