pytest-testinfra
pytest-testinfra copied to clipboard
pytest-dev
Question: is_installed was changed for Debian and an installed package cannot be found after the change
Hi, I'm finding differences running my molecules tests on Travis since the latest version of testinfra. My tests passed in 1.12, but now they are failing in 1.14.
I'm sure the package is installed in the machine, but the python tests says the opposite. Or maybe I'm wrong 🤔 Could you please help me with this issue?
Thanks!
Here I share my scenario:
Ansible task in my role:
- name: "Install jq and all GIT packages"
become: "yes"
apt:
name: "{{ item.name }}"
state: "{{ item.state }}"
changed_when: false
with_items:
- { name: "jq", state: "latest" }
- { name: "git-all", state: "latest" }
This is the output of the test execution:
TASK [jenkins-slave : Install jq and all GIT packages] *************************
ok: [instance] => (item={u'state': u'latest', u'name': u'jq'})
ok: [instance] => (item={u'state': u'latest', u'name': u'git-all'})
And the output of the idempotence check:
--> Action: 'idempotence'
Idempotence completed successfully.
This is the python test in molecule:
def test_git_all_is_installed(host):
assert host.package("git-all").is_installed
The output of the failing test result:
=================================== FAILURES ===================================
________________ test_git_all_is_installed[ansible://instance] _________________
host = <testinfra.host.Host object at 0x7fa2223b61d0>
def test_git_all_is_installed(host):
> assert host.package("git-all").is_installed
E AssertionError: assert False
E + where False = <package git-all>.is_installed
E + where <package git-all> = <class 'testinfra.modules.base.DebianPackage'>('git-all')
E + where <class 'testinfra.modules.base.DebianPackage'> = <testinfra.host.Host object at 0x7fa2223b61d0>.package
tests/test_default.py:29: AssertionError
------------------------------ Captured log call -------------------------------
ansible.py 61 INFO RUN Ansible(u'shell', u"dpkg-query -f '${Status}' -W git-all", {}): {'_ansible_no_log': False,
'_ansible_parsed': True,
u'changed': True,
u'cmd': u"dpkg-query -f '${Status}' -W git-all",
u'delta': u'0:00:00.042013',
u'end': u'2018-07-10 23:04:28.901809',
u'invocation': {u'module_args': {u'_raw_params': u"dpkg-query -f '${Status}' -W git-all",
u'_uses_shell': True,
u'chdir': None,
u'creates': None,
u'executable': None,
u'removes': None,
u'stdin': None,
u'warn': True}},
u'msg': u'non-zero return code',
u'rc': 1,
u'start': u'2018-07-10 23:04:28.859796',
u'stderr': u'dpkg-query: no packages found matching git-all',
'stderr_lines': [u'dpkg-query: no packages found matching git-all'],
u'stdout': u'',
'stdout_lines': []}
base.py 241 INFO RUN CommandResult(command=u"dpkg-query -f '${Status}' -W git-all", exit_status=1, stdout=u'', stderr=u'dpkg-query: no packages found matching git-all')
===================== 1 failed, 9 passed in 67.93 seconds ======================
By the way, I have other 4 tests where I check for packages using is_installed, and all of them works. So maybe it could be narrowed down to the git-all package...
By the way, using git instead of git-all fixed the issue, so I believe the problem is located in the git-all package
Hmm, very interesting, can you please provide the version of debian you are using? I just tested this on debian stretch and I do not get this error. I also wonder what version of ansible you are using, but I don't think this is a testinfra bug.
This is not the correct forum to discuss ansible or molecule problems but I suspect your problem might be in your playbook/role and not with testinfra itself. I could be wrong too but I currently cannot reproduce this error using the latest version of ansible or testinfra and I didn't even try to use molecule.
I suspect your problem might be that you have changed_when: false which I've never seen anyone do before on a package install and I can only assume this was a copy pasta mistake but again without knowing more details of your environment it's hard to say why you got this error.
Creating a symlink for the inventory
ln -s .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory inventory
cat test_packages.py
#!/usr/bin/env python
# import pytest
# import testinfra
import os
BASE_DIR = os.path.abspath(os.path.dirname(__file__))
INVENTORY = os.path.join(BASE_DIR, 'inventory')
testinfra_hosts = ('ansible://default?ansible_inventory=' + INVENTORY,)
def test_git_all_is_installed(host):
assert host.package("git-all").is_installed
cat Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.define 'default' do |config|
config.vm.box = "debian/stretch64"
config.vm.hostname = "localhost"
# Vagrant >= 1.7 wants to replace the insecure_key with public boxes, but
# there is a bug in that implentation so we just allow the insecure_key
# anyway.
config.ssh.insert_key = false
config.vm.box_check_update = false
config.vm.network "private_network", ip: "67.77.255.22"
config.vm.synced_folder ".", "/vagrant",
disabled: false,
type: "sshfs",
ssh_opts_append: "-o Compression=yes -o ControlPersist=60s -o ControlMaster=auto",
sshfs_opts_append: "-o cache=no -o nonempty"
# config.vm.provision "shell", path: "install.sh"
config.vm.provision "ansible" do |ansible|
ansible.verbose = "v"
ansible.playbook = "playbook.yml"
end
config.vm.provider "virtualbox" do |vb|
# # Display the VirtualBox GUI when booting the machine
# vb.gui = true
vb.cpus = "1"
vb.memory = "768"
vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
end
end
end
cat playbook.yml
---
- hosts: default
become: yes
tasks:
- name: "Install jq and all GIT packages"
become: "yes"
apt:
name: "{{ item.name }}"
state: "{{ item.state }}"
changed_when: false
with_items:
- { name: "jq", state: "latest" }
- { name: "git-all", state: "latest" }
when:
ansible_os_family == 'Debian'
cat ansible.cfg
[ssh_connection]
pipelining = True
[defaults]
inventory= ./inventory
error_on_missing_handler = True
ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}
deprecation_warnings = True
display_skipped_hosts = True
host_key_checking = False
gathering = smart
# gather_subset = all
fact_caching = jsonfile
fact_caching_connection = tmp/ansible-facts
fact_caching_timeout = 60
cat test-requirements.txt
ansible==2.6.1
asn1crypto==0.24.0
atomicwrites==1.1.5
attrs==18.1.0
bcrypt==3.1.4
cffi==1.11.5
cryptography==2.2.2
enum34==1.1.6
funcsigs==1.0.2
idna==2.7
ipaddress==1.0.22
Jinja2==2.10
MarkupSafe==1.0
more-itertools==4.2.0
paramiko==2.4.1
pluggy==0.6.0
py==1.5.4
pyasn1==0.4.3
pycparser==2.18
PyNaCl==1.2.1
pytest==3.6.3
PyYAML==3.13
six==1.11.0
testinfra==1.14.0
Create a new virtual env
virtualenv -p python2.7 venv
. venv/bin/activate
pip install -r test-requirements.txt
Run vagrant and provision the vm
vagrant up
# run the tests
$ pytest -vvs
=============================================================================================== test session starts ===============================================================================================
platform darwin -- Python 2.7.15, pytest-3.6.3, py-1.5.4, pluggy-0.6.0 -- /private/tmp/xenial-test/venv/bin/python2.7
cachedir: .pytest_cache
rootdir: /private/tmp/xenial-test, inifile:
plugins: testinfra-1.14.0
collected 1 item
test_packages.py::test_git_all_is_installed[ansible://default] INFO:testinfra:RUN Ansible(u'shell', u'uname -s', {}): {'_ansible_no_log': False,
'_ansible_parsed': True,
u'changed': True,
u'cmd': u'uname -s',
u'delta': u'0:00:00.002308',
u'end': u'2018-07-17 05:31:12.045883',
u'invocation': {u'module_args': {u'_raw_params': u'uname -s',
u'_uses_shell': True,
u'argv': None,
u'chdir': None,
u'creates': None,
u'executable': None,
u'removes': None,
u'stdin': None,
u'warn': True}},
u'rc': 0,
u'start': u'2018-07-17 05:31:12.043575',
u'stderr': u'',
'stderr_lines': [],
u'stdout': u'Linux',
'stdout_lines': [u'Linux']}
INFO:testinfra:RUN CommandResult(command=u'uname -s', exit_status=0, stdout=u'Linux', stderr=u'')
INFO:testinfra:RUN Ansible(u'shell', u'lsb_release -a', {}): {'_ansible_no_log': False,
'_ansible_parsed': True,
u'changed': True,
u'cmd': u'lsb_release -a',
u'delta': u'0:00:00.038283',
u'end': u'2018-07-17 05:31:12.282728',
u'invocation': {u'module_args': {u'_raw_params': u'lsb_release -a',
u'_uses_shell': True,
u'argv': None,
u'chdir': None,
u'creates': None,
u'executable': None,
u'removes': None,
u'stdin': None,
u'warn': True}},
u'rc': 0,
u'start': u'2018-07-17 05:31:12.244445',
u'stderr': u'No LSB modules are available.',
'stderr_lines': [u'No LSB modules are available.'],
u'stdout': u'Distributor ID:\tDebian\nDescription:\tDebian GNU/Linux 9.4 (stretch)\nRelease:\t9.4\nCodename:\tstretch',
'stdout_lines': [u'Distributor ID:\tDebian',
u'Description:\tDebian GNU/Linux 9.4 (stretch)',
u'Release:\t9.4',
u'Codename:\tstretch']}
INFO:testinfra:RUN CommandResult(command=u'lsb_release -a', exit_status=0, stdout=u'Distributor ID:\tDebian\nDescription:\tDebian GNU/Linux 9.4 (stretch)\nRelease:\t9.4\nCodename:\tstretch', stderr=u'No LSB modules are available.')
INFO:testinfra:RUN Ansible(u'shell', u'command -v dpkg-query', {}): {'_ansible_no_log': False,
'_ansible_parsed': True,
u'changed': True,
u'cmd': u'command -v dpkg-query',
u'delta': u'0:00:00.001788',
u'end': u'2018-07-17 05:31:12.467429',
u'invocation': {u'module_args': {u'_raw_params': u'command -v dpkg-query',
u'_uses_shell': True,
u'argv': None,
u'chdir': None,
u'creates': None,
u'executable': None,
u'removes': None,
u'stdin': None,
u'warn': True}},
u'rc': 0,
u'start': u'2018-07-17 05:31:12.465641',
u'stderr': u'',
'stderr_lines': [],
u'stdout': u'/usr/bin/dpkg-query',
'stdout_lines': [u'/usr/bin/dpkg-query']}
INFO:testinfra:RUN CommandResult(command=u'command -v dpkg-query', exit_status=0, stdout=u'/usr/bin/dpkg-query', stderr=u'')
INFO:testinfra:RUN Ansible(u'shell', u"dpkg-query -f '${Status}' -W git-all", {}): {'_ansible_no_log': False,
'_ansible_parsed': True,
u'changed': True,
u'cmd': u"dpkg-query -f '${Status}' -W git-all",
u'delta': u'0:00:00.010250',
u'end': u'2018-07-17 05:31:12.658868',
u'invocation': {u'module_args': {u'_raw_params': u"dpkg-query -f '${Status}' -W git-all",
u'_uses_shell': True,
u'argv': None,
u'chdir': None,
u'creates': None,
u'executable': None,
u'removes': None,
u'stdin': None,
u'warn': True}},
u'rc': 0,
u'start': u'2018-07-17 05:31:12.648618',
u'stderr': u'',
'stderr_lines': [],
u'stdout': u'install ok installed',
'stdout_lines': [u'install ok installed']}
INFO:testinfra:RUN CommandResult(command=u"dpkg-query -f '${Status}' -W git-all", exit_status=0, stdout=u'install ok installed', stderr=u'')
PASSED
Hey, thanks for such a detailed bugtrack 😊
My base OS is Ubuntu 16.04, this is something I did not add to the scenario. I set that at molecule.yml level:
---
dependency:
name: galaxy
driver:
name: docker
lint:
name: yamllint
platforms:
- name: instance
image: ubuntu:16.04
provisioner:
name: ansible
lint:
name: ansible-lint
scenario:
name: default
verifier:
name: testinfra
lint:
name: flake8
I tested this with molecule using your sample configuration and I still get passing tests. I'm fairly confident that the error you are getting is problems with your role + molecule configuration versus a bug with testinfra.
Perhaps you can create a gist of your example to show us how you have your environment setup so that we can try and reproduce the error using your gist?
I've personally tested this both with Vagrant, Docker and molecule and I cannot reproduce your error via testinfra. I get the error when the package is not installed which is the correct behavior of testinfra.
I'm just a contributor to testinfra and can say that I believe testinfra is behaving correctly in this case and wasn't in 1.12.0. Just my $2.