pytest-dev
testinfra conflicts with pytest-ansible
When both testinfra and pytest-ansible are installed, testinfra fails with the following output:
$ testinfra --connection=ansible --sudo --ansible-inventory=inventory test_proxy.py
Traceback (most recent call last):
File "/home/ivan/cde/devops/devops-env/bin/testinfra", line 11, in <module>
sys.exit(main())
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/testinfra/main.py", line 100, in main
return pytest.main()
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 38, in main
config = _prepareconfig(args, plugins)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 117, in _prepareconfig
pluginmanager=pluginmanager, args=args)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
_MultiCall(methods, kwargs, hook.spec_opts).execute()
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
return _wrapped_call(hook_impl.function(*args), self.execute)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
wrap_controller.send(call_outcome)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/helpconfig.py", line 28, in pytest_cmdline_parse
config = outcome.get_result()
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
_reraise(*ex) # noqa
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
self.result = func()
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
res = hook_impl.function(*args)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 852, in pytest_cmdline_parse
self.parse(args)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 957, in parse
self._preparse(args)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 922, in _preparse
self.known_args_namespace = ns = self._parser.parse_known_args(args)
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 490, in parse_known_args
return self.parse_known_and_unknown_args(args)[0]
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 496, in parse_known_and_unknown_args
optparser = self._getparser()
File "/home/ivan/cde/devops/devops-env/lib/python2.7/site-packages/_pytest/config.py", line 475, in _getparser
arggroup.add_argument(*n, **a)
File "/usr/lib64/python2.7/argparse.py", line 1308, in add_argument
return self._add_action(action)
File "/usr/lib64/python2.7/argparse.py", line 1509, in _add_action
action = super(_ArgumentGroup, self)._add_action(action)
File "/usr/lib64/python2.7/argparse.py", line 1322, in _add_action
self._check_conflict(action)
File "/usr/lib64/python2.7/argparse.py", line 1460, in _check_conflict
conflict_handler(action, confl_optionals)
File "/usr/lib64/python2.7/argparse.py", line 1467, in _handle_conflict_error
raise ArgumentError(action, message % conflict_string)
argparse.ArgumentError: argument --ansible-inventory: conflicting option string(s): --ansible-inventory
Removing --ansible-inventory from the command line does not change the output. Uninstalling pytest-ansible and running testinfra fixes the issue, but leaves my tests which use it broken.
Contents of test_proxy.py:
# Proxy servers must have nginx installed, running and enabled on boot
def test_nginx(Package, Service):
assert Package('nginx').is_installed
assert Service('nginx').is_running
assert Service('nginx').is_enabled
Environment: Python 2.7.11 pip 7.1.2 ansible 1.9.4 configured module search path = None virtualenv 13.1.2 testinfra 1.0.1 pytest-ansible 1.3.1 pytest 2.8.4
(testinfra command is an alias to py.test)
Both pytest-ansible and testinfra are pytest plugins that define the same --ansible-inventory option (pytest options scope is global). I'll look if there is a way to handle this in pytest.
BTW, I think you can easily rewrite your pytest-ansible tests with testinfra with the Ansible module: https://testinfra.readthedocs.org/en/latest/modules.html#ansible ;)
I'm using pytest-ansible to test custom modules and/or plugins, not the infrastructure itself (so it's not viable to just substitute it with testinfra). With testinfra I plan to rewrite all of my playbook's/role's unit tests (which are currently just special ansible playbooks) and expand infrastructure tests, if we can get it to work alongside pytest-ansible.
Let me know if I can help with anything.
As a workaround you can add -p no:pytest-ansible when running testinfra tests and -p no:testinfra when running pytest-ansible tests.
Is this still a valid issue? https://github.com/philpep/testinfra/issues/58#issuecomment-175734800 seems like a fine solution. That pytest-ansible package hasn't seen a release for quite some time ... wondering if this is relevant anymore.
I'm having a similar issue for a different arg - any ideas what I need to do?
python3.11 -m venv venv
source venv/bin/activate
pip install pytest
pip install pytest-ansible
pip install pytest-testinra
pip install ansible
pip list
Package Version
---------------- -------
ansible 8.4.0
ansible-core 2.15.4
cffi 1.16.0
coverage 7.3.2
cryptography 41.0.4
iniconfig 2.0.0
Jinja2 3.1.2
MarkupSafe 2.1.3
packaging 23.2
pip 23.2.1
pluggy 1.3.0
pycparser 2.21
pytest 7.4.2
pytest-ansible 4.1.0
pytest-testinfra 9.0.0
PyYAML 6.0.1
resolvelib 1.0.1
setuptools 68.1.2
snakeviz 2.2.0
tornado 6.3.3
pytest --collect-only
> argparse.ArgumentError: argument --connection: conflicting option string: --connection
pytest --collect-only -p no:pytest-ansible
> works
pytest --collect-only -p no:testinfra -p no:pytest-testinfra
> argparse.ArgumentError: argument --connection: conflicting option string: --connection
EDIT: Solved this via this stackoverflow answer: https://stackoverflow.com/a/65356704
Running python -c "import pkg_resources; print(' '.join('-p no:' + ' '.join(dist.get_entry_map(group='pytest11').keys()) for dist in pkg_resources.working_set if dist.get_entry_map(group='pytest11')))" gives you all the pytest plugin you have that you can disable.
pytest --collect-only -p no:pytest11.testinfra works
I've hit this issue too, and thanks to all the people that contributed to this thread I was able to achieve a solution, sharing it in case it is useful to someone else.
My problem
Using pytest-molecule for testing ansible roles in a monorepo, where each role is tested using molecule + testinfra.
This is an example layout
.
├── collections
│ └── ansible_collections
│ └── auburus
│ └── test_collection
│ ├── playbook.yml
│ └── roles
│ └── test_role
│ ├── molecule
│ │ └── default
│ │ ├── converge.yml
│ │ ├── molecule.yml
│ │ └── tests
│ │ └── test_role.py
│ └── tasks
│ └── main.yml
├── poetry.lock
└── pyproject.toml
In September pytest-molecule was archived, and the path forward is pytest-ansible, so the goal is to move to that one. But we couldn't move since pytest-ansible and pytest-testinfra got the conflict described in this issue.
Solution
Get each molecule test to ignore pytest ansible.
# molecule.yaml
...
verifier:
name: testinfra
options:
# pytest-ansible conflicts with --connection
p: "no:pytest-ansible"
# avoid loading global pyproject.toml options, so we don't
# load the setting that says "disable testinfra"
c: "."
At the top level, ignore pytest-testinfra
# pyproject.toml
[tool.pytest.ini_options]
norecursedirs="molecule"
# testinfra and pytest-ansible conflict as pytest
# plugins. We disable testinfra when running "all" tests,
# and disable `pytest-ansible` when running each molecule test
addopts = "-p no:pytest11.testinfra"
Finally follow this guide so all molecule tests are loaded when running pytest.
# tests/test_molecule.py
"""Tests for molecule scenarios."""
from __future__ import absolute_import, division, print_function
from pytest_ansible.molecule import MoleculeScenario
def test_integration(molecule_scenario: MoleculeScenario) -> None:
"""Run molecule for each scenario.
:param molecule_scenario: The molecule scenario object
"""
proc = molecule_scenario.test()
assert proc.returncode == 0
Results
- pytest can run all molecule tests from top level ✅
$ pytest --co
============================= test session starts =============================
platform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.3.0
ansible: 2.16.0
rootdir: /tmp/example-pytest-ansible-testinfra-conflict
configfile: pyproject.toml
plugins: ansible-4.1.1
collected 1 item
<Module tests/test_molecule.py>
<Function test_integration[test-default]>
============================= 1 test collected in 0.04s =============================
- each role can be tested with
molecule test✅
$ molecule test
...
WARNING Skipping, side effect playbook not configured.
INFO Running default > verify
INFO Executing Testinfra tests found in /tmp/example-pytest-ansible-testinfra-conflict/collections/ansible_collections/auburus/test/roles/test/molecule/default/tests/...
============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/example-pytest-ansible-testinfra-conflict/collections/ansible_collections/auburus/test/roles/test/molecule
configfile: default
plugins: testinfra-10.0.0
collected 1 item
tests/test_role.py . [100%]
============================== 1 passed in 0.70s ===============================
INFO Verifier completed successfully.
...
Edit: I've published this to a repo in case more details are needed
I ran into this over a year ago, but didn't have enough time to produce two documented PRs for the concurrent changes to both repos. I think pytest has a better way to deal with the conflict, which isn't at all easy to debug due to the way exception handling occurs when molecule calls pytest which dynamically loads plugins.
The TL;DR, both modules try to use the same command line options. My solution was to preface each option with a string specific to each module to deconflict. You can see the changes here:
https://github.com/ansible/pytest-ansible/compare/main...davedittrich:pytest-ansible:develop
https://github.com/pytest-dev/pytest-testinfra/compare/main...davedittrich:pytest-testinfra:develop
If someone else has time to resolve this, that would be great. :)
