No module named hvac after upgrading to ansible version 6.0.0 or 6.1.0 / module is running under python 2.7??!

[df-cgdm]

After upgrading ansible from version 5.10.0 to version 6.0.0 or 6.1.0 I'm getting the error:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: No module named hvac
fatal: [localhost]: FAILED! => {"changed": false, "failed_when_result": true, "module_stderr": "Traceback (most recent call last):\n  File \"/home/adm-dfournout/.ansible/tmp/ansible-tmp-1659342981.5344145-63417-27544398947102/AnsiballZ_hashivault_read.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/adm-dfournout/.ansible/tmp/ansible-tmp-1659342981.5344145-63417-27544398947102/AnsiballZ_hashivault_read.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/adm-dfournout/.ansible/tmp/ansible-tmp-1659342981.5344145-63417-27544398947102/AnsiballZ_hashivault_read.py\", line 48, in invoke_module\n    run_name='__main__', alter_sys=True)\n  File \"/usr/lib/python2.7/runpy.py\", line 188, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/lib/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/tmp/ansible_ansible.legacy.hashivault_read_payload_tZdTZc/ansible_ansible.legacy.hashivault_read_payload.zip/ansible/modules/hashivault/hashivault_read.py\", line 3, in <module>\n  File \"/tmp/ansible_ansible.legacy.hashivault_read_payload_tZdTZc/ansible_ansible.legacy.hashivault_read_payload.zip/ansible/module_utils/hashivault.py\", line 3, in <module>\nImportError: No module named hvac\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

What I'm seeing is the module is running with "python 2.7" but it should run under "python 3.9" (as it was with version 5.10.0).

Does someone has already used this module with ansible 6 (ansible core 2.13) ? Is there any configuration to be done on ansible 6 to make it works ?

[df-cgdm]

The problem seems to come from the "shebang" in the modules. When I remove the shebang from "hashivault_read.py", it's working. Replacing the shebang "#!/usr/bin/env python" by "#!/usr/bin/python" is also working.

[TerryHowe]

What happens when you run /usr/bin/env python ? no such file or directory?

What distro are you running on?

[df-cgdm]

I'm running on a Debian 11. The problem is that /usr/bin/env python run python 2.7 rather than python 3.9 because it's the system default. The hvac library is only installed with python 3.9. When the shebang is /usr/bin/python Ansible handle it as a standard "python" module and it's using the latest installed python version. When the shebang is /usr/bin/env python it use the shebang to run the module. This behavior seems new to Ansible 6 (ansible core 2.13)

[df-cgdm]

The ansible documentation (https://docs.ansible.com/ansible/latest/dev_guide/testing/sanity/shebang.html) is saying

"This does not apply to Ansible modules, which should not be executable and must always use #!/usr/bin/python."

[pat-s]

I am seeing potentially similar issues on Alpine 3.16 (which removed Python2):

task path: /drone/src/helper-tasks/pre-tasks-onboard.yml:37
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "env: can't execute 'python': No such file or directory\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 127}

/ansible # /usr/bin/env python
env: can't execute 'python': No such file or directory

I could fix it by

ln -s /usr/bin/python3 /usr/bin/python

[pat-s]

An update on this, as my colleague and me just fell over this in another instance.

I had v4.6.6 installed, my colleague 4.7.0.

• With 4.6.6, /usr/bin/python3.6 was used for the module call (which is the default interpreter that was discovered by ansible on the remote).
• With 4.7.0, /usr/bin/env python was used which on our remote defaults to python2.7

Because of the new behavior in 4.7.0, hvac python module was not installed as our pip call in the pipeline only does so for the default pip on the system (which defaults to python3.6).

To workaround this for the moment, we added another pip call to the default pip2 interpreter (if available).

Overall, I think this module should just use the default ansible python interpreter used in all other tasks rather than hardcoding it's one to /usr/bin/env python which might differ to the default one used by ansible. Also there are still many systems out there for which /usr/bin/env python points to Python2 and still using it might incur other side issues.

Thanks for maintaining this module!

[pat-s]

In case it helps somebody here, our current configuration to account for the installation of hvac across various OS while accounting for the python2/python3 mess:

• Check for existence of Python2 (we assume /usr/bin/env python links to python2).
  • If python2 does not exist, symlink to python3
  • If exists, check for existence of pip2.
    • If pip2 is missing, install pip2
    • If pip2 exists, install hvac
• Always install hvac for python3

- name: Check that /usr/bin/env python exists (for hashivault modules)
  stat:
    path: /usr/bin/python
  register: python_exists

- name: Symlink /usr/bin/env python to /usr/bin/env python3
  file:
    src: "/usr/bin/python3"
    dest: "/usr/bin/python"
    state: link
  when: not python_exists.stat.exists

- name: Check if pip2 exists
  stat:
    path: /usr/bin/pip2
  register: pip2_exists

- name: Ensure pip hvac is installed if python2 exists
  when: pip2_exists.stat.exists
  pip:
    executable: /usr/bin/pip2
    name:
      - hvac
      - hvac[parser]
    state: present

- name: Install hvac pip required for hashicorp_vault module
  become: true
  become_user: "{{ user }}"
  pip:
    name:
      - hvac
      - hvac[parser]
    state: latest

[pavel-z1]

We have the same issue /usr/bin/env: ‘python’: No such file or directory Used Rocky Linux 8.7 and ansible [core 2.14.7]

MacOS 13.4.1 - the same error

[TerryHowe]

You'll need to make sure some Python version is in your path when you are running

[pavel-z1]

Of course can be created a symlink But maybe a better solution would be replace the shebang "#!/usr/bin/env python" by "#!/usr/bin/python" as recommend ansible docs https://docs.ansible.com/ansible/latest/dev_guide/testing/sanity/shebang.html ?

[ceesios]

Agree,

As described here: "Begin your Ansible module with #!/usr/bin/python - this “shebang” allows ansible_python_interpreter to work. Follow the shebang immediately with # -- coding: utf-8 -- to clarify that the file is UTF-8 encoded."

"Using #!/usr/bin/env, makes env the interpreter and bypasses ansible__interpreter logic."

I am running into this issue when trying to use the modules in an pre-built k8s operator image.

This should do the trick for i in /.py; do sed -i 's/#!/usr/bin/env python/#!/usr/bin/python/g' $i; done

[GregWhiteyBialas]

I have created PR to solve the issue. Any kind of comments/help/contribution is welcomed.

[TerryHowe]

I think the idea was to change the shebang, not remove it.

[nyetwurk]

Creating and maintaining a non-pip based py-hvac/py3-hvac for alpine and a python-hvac/python3-hvac for debian would solve this for both cases.

pip combined with venv just creates more problems than either attempt to solve - imo both should be avoided in everything but a dev context.

[nyetwurk]

Insisting on venv requires --system-site-packages but still breaks hardcoded shebangs.

Eschewing venv (so you dont have to use env python) allows hardcoded shebangs, but requires PIP_REQUIRE_VIRTUALENV=false and possibly --break-system-packages as well.

The whole situation is a mess introduced by venv which allowed folks to use very questionable workflows - all because pip+ venv suddenly allowed developers to not bother to create native distro packaging for their python modules.