ansible-lint icon indicating copy to clipboard operation
ansible-lint copied to clipboard
verified publisher icon ansible

ansible-lint doesn't make use of collection installed for the first time

Open cla86 opened this issue 2 years ago • 3 comments

Summary

ansible-lint doesn't make use of collection installed for the first time

Issue Type
  • Bug Report
OS / ENVIRONMENT
ansible-lint 6.17.2 using ansible-core:2.15.2 ansible-compat:4.1.5 ruamel-yaml:0.17.32 ruamel-yaml-clib:0.2.7

This is docker image setup for CI/CD pipeline based on docker.io/library/python:3.11.4-bullseye only with ansible-core and ansible-lint

  • ansible installation method: pip
  • ansible-lint installation method: pip
STEPS TO REPRODUCE
Desired Behavior

ansible-lint should analyze ansible content upon installed collections.

Possible security bugs should be reported via email to [email protected]

Actual Behavior

Running ansible-lint for the first time it installs collections from requirements.yml file, but doesn't make use of it producing warnings about not being able to resolve module from collections.

root@c58481c9f5c5:/data# ansible-lint -v main.yml
INFO     Identified /data as project root due .git directory.
INFO     Running ansible-galaxy collection install -v -r requirements.yml
INFO     Set ANSIBLE_LIBRARY=/root/.cache/ansible-compat/3a6eb0/modules:/root/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/root/.cache/ansible-compat/3a6eb0/collections:/root/.cache/ansible-compat/3a6eb0/collections:/root/.ansible/collections:/usr/share/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/root/.cache/ansible-compat/3a6eb0/roles:roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
WARNING  Skipped installing collection dependencies due to running in offline mode.
INFO     Set ANSIBLE_LIBRARY=/root/.cache/ansible-compat/3a6eb0/modules:/root/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/root/.cache/ansible-compat/3a6eb0/collections:/root/.ansible/collections:/usr/share/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/root/.cache/ansible-compat/3a6eb0/roles:roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
INFO     Executing syntax check on playbook main.yml (4.82s)
WARNING  Unable to load module community.postgresql.postgresql_query at roles/postgres/handlers/main.yml:25 for options validation
WARNING  Unable to load module community.postgresql.postgresql_query at roles/postgres/handlers/main.yml:36 for options validation
WARNING  Unable to load module community.postgresql.postgresql_query at roles/postgres/handlers/main.yml:47 for options validation
WARNING  Unable to load module community.postgresql.postgresql_query at roles/postgres/handlers/main.yml:58 for options validation
WARNING  Unable to resolve FQCN for module community.postgresql.postgresql_query
WARNING  Unable to load module community.postgresql.postgresql_user at roles/repmgr/tasks/main.yml:59 for options validation
WARNING  Unable to load module community.postgresql.postgresql_user at roles/repmgr/tasks/main.yml:71 for options validation
WARNING  Unable to load module community.postgresql.postgresql_db at roles/repmgr/tasks/main.yml:86 for options validation
WARNING  Unable to load module community.postgresql.postgresql_db at roles/repmgr/tasks/main.yml:96 for options validation
WARNING  Unable to resolve FQCN for module community.postgresql.postgresql_user
WARNING  Unable to resolve FQCN for module community.postgresql.postgresql_db
WARNING  Unable to load module community.general.sefcontext at roles/uwsgi/tasks/main.yml:79 for options validation
WARNING  Unable to resolve FQCN for module community.general.sefcontext
WARNING  Unable to load module community.general.ini_file at roles/dns/tasks/main.yml:2 for options validation
WARNING  Unable to resolve FQCN for module community.general.ini_file
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:42 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:60 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:101 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:109 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:119 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:130 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:141 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:152 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:163 for options validation
WARNING  Unable to load module ansible.posix.firewalld at roles/firewalld/tasks/main.yml:174 for options validation
WARNING  Unable to resolve FQCN for module ansible.posix.firewalld
WARNING  Unable to load module ansible.posix.authorized_key at roles/pki/tasks/main.yml:3 for options validation
WARNING  Unable to resolve FQCN for module ansible.posix.authorized_key
[DEPRECATION WARNING]: Encryption using the Python crypt module is deprecated. The Python crypt module is deprecated and will be removed from Python 3.13. Install the passlib library for continued encryption functionality. This feature
will be removed in version 2.17. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
WARNING  Unable to load module community.general.timezone at roles/localtime/tasks/main.yml:2 for options validation
WARNING  Unable to resolve FQCN for module community.general.timezone
WARNING  Unable to load module ansible.posix.mount at roles/nfs-mount/tasks/main.yml:13 for options validation
WARNING  Unable to resolve FQCN for module ansible.posix.mount
WARNING  Unable to load module community.rabbitmq.rabbitmq_user at roles/rabbitmq/tasks/main.yaml:71 for options validation
WARNING  Unable to resolve FQCN for module community.rabbitmq.rabbitmq_user

Passed: 0 failure(s), 0 warning(s) on 53 files. Last profile that met the validation criteria was 'production'.

Running ansible-lint second time works as expected

root@c58481c9f5c5:/data# ansible-lint -v main.yml
INFO     Identified /data as project root due .git directory.
INFO     Running ansible-galaxy collection install -v -r requirements.yml
INFO     Set ANSIBLE_LIBRARY=/root/.cache/ansible-compat/3a6eb0/modules:/root/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/root/.cache/ansible-compat/3a6eb0/collections:/root/.cache/ansible-compat/3a6eb0/collections:/root/.ansible/collections:/usr/share/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/root/.cache/ansible-compat/3a6eb0/roles:roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
WARNING  Skipped installing collection dependencies due to running in offline mode.
INFO     Set ANSIBLE_LIBRARY=/root/.cache/ansible-compat/3a6eb0/modules:/root/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/root/.cache/ansible-compat/3a6eb0/collections:/root/.ansible/collections:/usr/share/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/root/.cache/ansible-compat/3a6eb0/roles:roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
INFO     Executing syntax check on playbook main.yml (3.78s)
[DEPRECATION WARNING]: Encryption using the Python crypt module is deprecated. The Python crypt module is deprecated and will be removed from Python 3.13. Install the passlib library for continued encryption functionality. This feature
will be removed in version 2.17. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

Passed: 0 failure(s), 0 warning(s) on 53 files. Last profile that met the validation criteria was 'production'.
root@c58481c9f5c5:/data#

cla86 avatar Jul 27 '23 09:07 cla86

I think this issue makes the ansible-lint GitHub action unusable if you are using any module outside Ansible core. There's no opportunity to run ansible-galaxy by hand between the installation and execution of ansible-lint, so you need to rely on its implicit processing of requirements.yml to download any dependencies. Is the only workaround to bypass the action and install the dependencies and linter yourself?

ursetto avatar Nov 03 '23 02:11 ursetto

I can confirm this happening in latest ansible-lint version 24.6.0 too. Using ansible version 2.17.0.

Workaround is to run ansible-galaxy install -r requirements.yml before running ansible-lint.

mhanc avatar Jun 20 '24 11:06 mhanc

I think this issue makes the ansible-lint GitHub action unusable if you are using any module outside Ansible core. There's no opportunity to run ansible-galaxy by hand between the installation and execution of ansible-lint, so you need to rely on its implicit processing of requirements.yml to download any dependencies. Is the only workaround to bypass the action and install the dependencies and linter yourself?

In Issue #3942 user veksh also showed an alternative solution for this when using the GitHub Action, which was to do as you suggested. Since @cla86 found that running it a second time works, I tested that in my GitHub workflow with success. Is it faster than writing a command to update packages, install pip, install Ansible, and then running the command? Probably, I didn't try to be honest, but this is working well for me now in my runners. We know that a second run will work, and all it takes is running ansible-lint once to trigger it to download any required collections and roles. In order to make the first ansible-lint action go faster, I am passing an argument to only look at a single file. On the second pass it's running against the entire repository.

jobs:
  build:
    name: Ansible Lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      # This runs twice, once to install the collections and roles, because
      # the current action is unable to resolve FQCN for modules on first pass
      - name: Install required collections and roles
        uses: ansible/ansible-lint@v24
        with:
          args: "collections/requirements.yml -v"
      - name: Run ansible-lint
        uses: ansible/ansible-lint@v24
        with:
          args: "-v"

barstown avatar Jun 28 '24 19:06 barstown

Hey folks,

I've been looking into this as I noticed when using the GitHub action, ansible-lint was showing errors due to the roles and collections defined in my requirements.yml file not being installed.

On my fork, I've implemented a new optional input for the action called requirements_file which can be provided a path to a requirements file containing roles and collections to be installed prior to running ansible-lint: https://github.com/dbrennand/ansible-lint/commit/f005e8d06975be8c327a6341cc51d3acb026b9df

Usage example: https://github.com/dbrennand/home-ops/blob/dev/.github/workflows/ansible-lint.yml#L15 Sample run: https://github.com/dbrennand/home-ops/actions/runs/9879791162/job/27286802492

I'll be raising a PR for this shortly 🙂

dbrennand avatar Jul 10 '24 18:07 dbrennand