3.0 breaks connections using ansible (part 2)

[mrdima]

Version 3.0 (all from .0.0 to .0.5) breaks connections to ansible hosts behind a "jumphost or bastionhost". (#439 Doesn't fix it) What worked fine in 2.x version now doesn't. For some reason testinfra wants to lookup host dns names locally, but that is in our case impossible. They can only be resolved when using the jumphost.

Consider the following inventory:

[all:vars]
ansible_ssh_user           = myuser
ansible_python_interpreter = /opt/pypy/bin/pypy
ansible_ssh_common_args    = "-o StrictHostKeyChecking=no -o ProxyCommand=\"ssh -o StrictHostKeyChecking=no -W %h:%p -q {{ansible_ssh_user}}@{{jumphost}} -p 12345\""
jumphost                   = 123.123.1.1


[jumphost]
123.123.1.1:12345
[nodes]
da0-n-8ca8
da0-n-2bc3
da0-n-1ee6

Which basically says (using the common_args) that it needs to reach everything using the jumphost ssh running on port 12345 Then running: py.test --color=yes -n 3 -vv --hosts='ansible://nodes' --ansible-inventory /inventory -p no:cacheprovider

Results in:

platform linux -- Python 3.7.3, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python3.7
rootdir: /tests
plugins: forked-1.0.2, xdist-1.29.0, testinfra-3.0.5
[gw0] linux Python 3.7.3 cwd: /tests
[gw1] linux Python 3.7.3 cwd: /tests
[gw2] linux Python 3.7.3 cwd: /tests
[gw0] Python 3.7.3 (default, May  3 2019, 11:24:39)  -- [GCC 8.3.0]
[gw1] Python 3.7.3 (default, May  3 2019, 11:24:39)  -- [GCC 8.3.0]
[gw2] Python 3.7.3 (default, May  3 2019, 11:24:39)  -- [GCC 8.3.0]
gw0 [3] / gw1 [3] / gw2 [3]
scheduling tests via LoadScheduling

test_t.py::test_update_settings[ansible://da0-n-8ca8] 
test_t.py::test_update_settings[ansible://da0-n-2bc3] 
test_t.py::test_update_settings[ansible://da0-n-1ee6] 
[gw0] [ 33%] FAILED test_t.py::test_update_settings[ansible://da0-n-1ee6] 
[gw1] [ 66%] FAILED test_t.py::test_update_settings[ansible://da0-n-2bc3] 
[gw2] [100%] FAILED test_t.py::test_update_settings[ansible://da0-n-8ca8] 

.....

______________________________________________________ test_update_settings[ansible://da0-n-8ca8] _______________________________________________________
[gw2] linux -- Python 3.7.3 /usr/bin/python3.7

host = <testinfra.host.Host object at 0x7f40b5266358>

    def test_update_settings(host):
    # update settings (use not instead of == False??). Also, probably assign the service to a variable instead of "checking" twice or more?!
>       assert host.service("update-engine").is_running == False

test_t.py:8: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.7/site-packages/testinfra/host.py:107: in __getattr__
    obj = module_class.get_module(self)
/usr/lib/python3.7/site-packages/testinfra/modules/base.py:22: in get_module
    klass = cls.get_module_class(_host)
/usr/lib/python3.7/site-packages/testinfra/modules/service.py:51: in get_module_class
    if host.system_info.type == "linux":
/usr/lib/python3.7/site-packages/testinfra/modules/systeminfo.py:143: in type
    return self.sysinfo["type"]
/usr/lib/python3.7/site-packages/testinfra/utils/__init__.py:44: in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
/usr/lib/python3.7/site-packages/testinfra/modules/systeminfo.py:33: in sysinfo
    uname = self.run_expect([0, 1], 'uname -s')
/usr/lib/python3.7/site-packages/testinfra/host.py:71: in run
    return self.backend.run(command, *args, **kwargs)
/usr/lib/python3.7/site-packages/testinfra/backend/ansible.py:46: in run
    ssh_identity_file=self.ssh_identity_file)
/usr/lib/python3.7/site-packages/testinfra/utils/ansible_runner.py:179: in run
    return self.get_host(host, **kwargs).run(command)
/usr/lib/python3.7/site-packages/testinfraVersion 3.0 (all from .0.0 to .0.5) breaks connections to ansible hosts behind a "jumphost or bastionhost".
What worked fine in 2.x version now doesn't. For some reason testinfra wants to lookup host dns names locally, but that is in our case impossible. They can only be resolved when using the jumphost.

Consider the following inventory:
[all:vars]
ansible_ssh_user           = myuser
ansible_python_interpreter = /opt/pypy/bin/pypy
ansible_ssh_common_args    = "-o StrictHostKeyChecking=no -o ProxyCommand=\"ssh -o StrictHostKeyChecking=no -W %h:%p -q {{ansible_ssh_user}}@{{jumphost}} -p 12345\""
jumphost                   = 123.123.1.1


[jumphost]
123.123.1.1:12345
[nodes]
da0-n-8ca8
da0-n-2bc3
da0-n-1ee6/host.py:71: in run
    return self.backend.run(command, *args, **kwargs)
/usr/lib/python3.7/site-packages/testinfra/backend/paramiko.py:117: in run
    rc, stdout, stderr = self._exec_command(command)
/usr/lib/python3.7/site-packages/testinfra/backend/paramiko.py:104: in _exec_command
    chan = self.client.get_transport().open_session()
/usr/lib/python3.7/site-packages/testinfra/utils/__init__.py:44: in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
/usr/lib/python3.7/site-packages/testinfra/backend/paramiko.py:100: in client
    client.connect(**cfg)
/usr/lib/python3.7/site-packages/paramiko/client.py:340: in connect
    to_try = list(self._families_and_addresses(hostname, port))
/usr/lib/python3.7/site-packages/paramiko/client.py:204: in _families_and_addresses
    hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

host = 'da0-n-8ca8', port = 22, family = <AddressFamily.AF_UNSPEC: 0>, type = <SocketKind.SOCK_STREAM: 1>, proto = 0, flags = 0

    def getaddrinfo(host, port, family=0, type=0, proto=0, flags=0):
        """Resolve host and port into list of address info entries.
    
        Translate the host/port argument into a sequence of 5-tuples that contain
        all the necessary arguments for creating a socket connected to that service.
        host is a domain name, a string representation of an IPv4/v6 address or
        None. port is a string service name such as 'http', a numeric port number or
        None. By passing None as the value of host and port, you can pass NULL to
        the underlying C API.
    
        The family, type and proto arguments can be optionally specified in order to
        narrow the list of addresses returned. Passing zero as a value for each of
        these arguments selects the full range of results.
        """
        # We override this function since we want to translate the numeric family
        # and socket type values to enum constants.
        addrlist = []
>       for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
E       socket.gaierror: [Errno -2] Name does not resolve

As stated, this works fine in testinfra 2.x. How can we get this to work with 3.x?

Thanks!

[philpep]

I think testinfra should take in account ansible_ssh_common_args and jumphost from inventory.

Or maybe have a way to disable direct connection with testinfra ssh backend and connect directly with ansible (this is slower because it rely on a subprocess call to ansible, but all ansible connections options will work).

[mrdima]

I think both options would be cool, eg a parameter to use either ansible or direct. Via ansible is currently really slow indeed, so direct connection would be great, but indeed the args should be taken into account then.