ansible
Local collection install conflict between ansible-lint and molecule
Summary
It appears there is a conflict with how molecule and ansible-lint install a project if it is a collection.
Molecule uses ansible-galaxy to install the collection in the ansible-compat cache directory, while ansible-lint attempts to create a symlink to the collection in the same cache directory. If you run molecule first, ansible-lint will fail and conversely if you run ansible-lint first, molecule will fail.
Issue Type
- Bug Report
Ansible and Ansible Lint details
$ ansible --version
ansible [core 2.13.2]
config file = None
configured module search path = ['/home/alde/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/alde/example/venv/lib/python3.10/site-packages/ansible
ansible collection location = /home/alde/.ansible/collections:/usr/share/ansible/collections
executable location = /home/alde/example/venv/bin/ansible
python version = 3.10.0rc2 (default, May 1 2022, 08:45:48) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)]
jinja version = 3.1.2
libyaml = True
$ ansible-lint --version
ansible-lint 6.4.1.dev5 using ansible 2.13.2
$ molecule --version
molecule 4.0.1 using python 3.10
ansible:2.13.2
delegated:4.0.1 from molecule
podman:2.0.2 from molecule_podman requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
- ansible installation method: pip
- ansible-lint installation method: source, main branch
OS / ENVIRONMENT
- Fedora 33
- Python 3.10
STEPS TO REPRODUCE
$ ansible-galaxy collection init dummy.example
$ cd dummy/example
$ molecule init scenario
$ molecule create
$ ansible-lint
Traceback (most recent call last):
File "/home/alde/example/venv/bin/ansible-lint", line 8, in <module>
sys.exit(_run_cli_entrypoint())
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/__main__.py", line 266, in _run_cli_entrypoint
sys.exit(main(sys.argv))
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/__main__.py", line 166, in main
app = get_app(offline=options.offline)
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/app.py", line 260, in get_app
_perform_mockings()
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/_mockings.py", line 92, in _perform_mockings
while os.readlink(link_path) != target:
OSError: [Errno 22] Invalid argument: '/home/alde/.cache/ansible-compat/50d858/collections/ansible_collections/dummy/example'
Desired Behavior
I expect ansible-lint to provide lint results.
Actual Behavior
Ansible lint fails because /home/alde/.cache/ansible-compat/50d858/collections/ansible_collections/dummy/example exists
from prior molecule run and is not a symlink.
$ ansible-lint
Traceback (most recent call last):
File "/home/alde/example/venv/bin/ansible-lint", line 8, in <module>
sys.exit(_run_cli_entrypoint())
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/__main__.py", line 266, in _run_cli_entrypoint
sys.exit(main(sys.argv))
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/__main__.py", line 166, in main
app = get_app(offline=options.offline)
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/app.py", line 260, in get_app
_perform_mockings()
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/_mockings.py", line 92, in _perform_mockings
while os.readlink(link_path) != target:
OSError: [Errno 22] Invalid argument: '/home/alde/.cache/ansible-compat/50d858/collections/ansible_collections/dummy/example'
Erroring in this case is expected but I agree that a stack trace is far from ideal.
The linter would not override symlinks with mocks in order to prevent accidents. If someone created a folder there, it means that you don't need mocking. Mocking will work only when there is nobody else creating overlapping data.
There are clear indications regarding dangers of using mocking and we discourage it, the entire feature was added more as a temporary measure for those that are unable to gather their dependencies. Mocks are removed after run, but we never want to remove non-mocks, so you can probably guess how use of mocks could end-up removing data from disk.
I will make a change to avoid a stack trace, but we will not be able to make it work. Best direction for you is to remove the mocks.
@aldehir I just merged a change that might sort the issue for you. Can you please try the code from main branch and let me know?
@ssbarnea Unfortunately I am still getting an error, different one this time however.
$ ansible-lint
Traceback (most recent call last):
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/_mockings.py", line 98, in _perform_mockings
if os.readlink(link_path) != target:
OSError: [Errno 22] Invalid argument: '/home/alde/.cache/ansible-compat/50d858/collections/ansible_collections/dummy/example'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/alde/example/venv/bin/ansible-lint", line 8, in <module>
sys.exit(_run_cli_entrypoint())
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/__main__.py", line 266, in _run_cli_entrypoint
sys.exit(main(sys.argv))
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/__main__.py", line 166, in main
app = get_app(offline=options.offline)
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/app.py", line 260, in get_app
_perform_mockings()
File "/home/alde/example/venv/lib/python3.10/site-packages/ansiblelint/_mockings.py", line 102, in _perform_mockings
link_path.unlink(missing_ok=True)
File "/home/alde/.pyenv/versions/3.10.0rc2/lib/python3.10/pathlib.py", line 1204, in unlink
self._accessor.unlink(self)
IsADirectoryError: [Errno 21] Is a directory: '/home/alde/.cache/ansible-compat/50d858/collections/ansible_collections/dummy/example'
Molecule installs the collection to ~/.cache/ansible-compat/<hash>/collections/ansible_collections/dummy/example, so the directory for the collection exists but since it is an entire directory structure and not a symlink, os.readlink() fails followed by unlink() failing for the same reason.
Regarding your advice to remove mocks, I am using a minimal example without any mocks that I configured myself.
I am going to close this due to lack of reproducer on it. If you can please create a small repository with a script at the root that reproduces that error we can work on addressing it.
@ssbarnea I've been struggling with the same issue, so I've created a small repo with an example collection and a Makefile with targets to reproduce the bug. The repo is here: https://github.com/heindsight/ansible-lint-vs-molecule.git
The bug can be reproduced by doing
git clone https://github.com/heindsight/ansible-lint-vs-molecule.git
cd ansible-lint-vs-molecule
make venv
. venv/bin/activate
make molecule-then-lint
To demonstrate that a similar issue also occurs when running ansible-lint first, followed by molecule you can also run
make lint-then-molecule
The make targets will override the XDG_CACHE_HOME env var to avoid polluting your ~/.cache (and to avoid any interference between the two reproduction scenarios). Incidentally, this is also how I currently work around the bug - I set XDG_CACHE_HOME to different values when running molecule or ansible-lint.
