community.crypto
community.crypto copied to clipboard
Published
20 hours ago •
ansible-collections
ansible-collections
community.crypto.openssl_privatekey module also needs 'cffi'
SUMMARY
The community.crypto.openssl_privatekey module also requires the cffi package, however the documentation only lists cryptography as a dependency.
ISSUE TYPE
- Documentation Report
COMPONENT NAME
community.crypto.openssl_privatekey
ANSIBLE VERSION
ansible [core 2.17.4]
config file = /home/jvilla/projects/path-production-ansible/ansible.cfg
configured module search path = ['/home/jvilla/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/jvilla/.local/pipx/venvs/ansible-core/lib/python3.10/site-packages/ansible
ansible collection location = /home/jvilla/.ansible/collections:/usr/share/ansible/collections
executable location = /home/jvilla/.local/bin/ansible
python version = 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] (/home/jvilla/.local/pipx/venvs/ansible-core/bin/python)
jinja version = 3.1.4
libyaml = True
Some more details:
I am calling the community.crypto.openssl_privatekey module, and it fails because it is missing the "cffi backend" module.
Here is the -vvv output of this failing task:
ansible-playbook -vvv output
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: jvilla
<localhost> EXEC /bin/sh -c 'echo ~jvilla && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/jvilla/.ansible/tmp `"&& mkdir "` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243 `" && echo ansible-tmp-1727964631.7938662-12315-25707778985243="` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243 `" ) && sleep 0'
Using module file /home/jvilla/.ansible/collections/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py
<localhost> PUT /home/jvilla/.ansible/tmp/ansible-local-121319ffj6jk5/tmp8rv9m_a5 TO /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/ /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3.11 /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py", line 107, in <module>
_ansiballz_main()
File "/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py", line 99, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py", line 47, in invoke_module
runpy.run_module(mod_name='ansible_collections.community.crypto.plugins.modules.openssl_privatekey', init_globals=dict(_module_fqn='ansible_collections.community.crypto.plugins.modules.openssl_privatekey', _modlib_path=modlib_path),
File "<frozen runpy>", line 226, in run_module
File "<frozen runpy>", line 98, in _run_module_code
File "<frozen runpy>", line 88, in _run_code
File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py", line 301, in <module>
File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py", line 281, in main
File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/privatekey.py", line 498, in select_backend
File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/privatekey.py", line 301, in __init__
File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/__init__.py", line 14, in default_backend
from cryptography.hazmat.backends.openssl.backend import backend
File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/__init__.py", line 6, in <module>
from cryptography.hazmat.backends.openssl.backend import backend
File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 114, in <module>
from cryptography.hazmat.bindings.openssl import binding
File "/usr/lib/python3/dist-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
from cryptography.hazmat.bindings._openssl import ffi, lib
ModuleNotFoundError: No module named '_cffi_backend'
fatal: [localhost]: FAILED! => {
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File \"/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py\", line 107, in <module>\n _ansiballz_main()\n File \"/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py\", line 99, in _ansiballz_main\n in
voke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py\", line 47, in invoke_module\n runpy.run_module(mod_name='ansible_collections.community.crypto.plugins.modules.openssl_privatekey', init_globals=dict(_module_fqn='ansible_collections.community.crypto.plugins
.modules.openssl_privatekey', _modlib_path=modlib_path),\n File \"<frozen runpy>\", line 226, in run_module\n File \"<frozen runpy>\", line 98, in _run_module_code\n File \"<frozen runpy>\", line 88, in _run_code\n File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypt
o/plugins/modules/openssl_privatekey.py\", line 301, in <module>\n File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py\", line 281, in main\n File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_commun
ity.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/privatekey.py\", line 498, in select_backend\n File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/pr
ivatekey.py\", line 301, in __init__\n File \"/usr/lib/python3/dist-packages/cryptography/hazmat/backends/__init__.py\", line 14, in default_backend\n from cryptography.hazmat.backends.openssl.backend import backend\n File \"/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/__init__.py\", line 6, in <module>\n from cryptography.hazmat.backends.openssl.bac
kend import backend\n File \"/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/backend.py\", line 114, in <module>\n from cryptography.hazmat.bindings.openssl import binding\n File \"/usr/lib/python3/dist-packages/cryptography/hazmat/bindings/openssl/binding.py\", line 14, in <module>\n from cryptography.hazmat.bindings._openssl import ffi, lib\nModuleNot
FoundError: No module named '_cffi_backend'\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}
Per that output, it is executing /usr/bin/python3.11 on the host, which at the time of this error had the following packages installed:
pip list, when the error occurred
$ /usr/bin/python3.11 -m pip list
Package Version
------------------- -------------
argcomplete 1.8.1
attrs 21.2.0
Automat 20.2.0
Babel 2.8.0
bcrypt 3.2.0
blinker 1.4
certifi 2020.6.20
chardet 4.0.0
click 8.0.3
cloud-init 24.3.1
colorama 0.4.4
command-not-found 0.3
configobj 5.0.6
constantly 15.1.0
cryptography 3.4.8
dbus-python 1.2.18
distro 1.7.0
distro-info 1.1+ubuntu0.2
httplib2 0.20.2
hyperlink 21.0.0
idna 3.3
importlib-metadata 4.6.4
incremental 21.3.0
jeepney 0.7.1
Jinja2 3.0.3
jsonpatch 1.32
jsonpointer 2.0
jsonschema 3.2.0
keyring 23.5.0
launchpadlib 1.10.16
lazr.restfulclient 0.14.4
lazr.uri 1.0.6
livereload 2.6.3
Markdown 3.3.6
MarkupSafe 2.0.1
mkdocs 1.1.2
more-itertools 8.10.0
netifaces 0.11.0
oauthlib 3.2.0
packaging 21.3
pip 22.0.2
pipx 1.0.0
psutil 5.9.0
pyasn1 0.4.8
pyasn1-modules 0.2.1
pycurl 7.44.1
Pygments 2.11.2
PyGObject 3.42.1
PyHamcrest 2.0.2
pyinotify 0.9.6
PyJWT 2.3.0
pyOpenSSL 21.0.0
pyparsing 2.4.7
pyrsistent 0.18.1
pyserial 3.5
python-apt 2.4.0+ubuntu4
pytz 2022.1
PyYAML 5.4.1
requests 2.25.1
SecretStorage 3.3.1
service-identity 18.1.0
setuptools 59.6.0
six 1.16.0
systemd-python 234
tornado 6.1
Twisted 22.1.0
ubuntu-pro-client 8001
ufw 0.36.1
unattended-upgrades 0.1
urllib3 1.26.5
userpath 1.8.0
wadllib 1.3.6
wheel 0.37.1
zipp 1.0.0
zope.interface 5.4.0
After installing cffi (output below), the task succeeded:
pip install cffi, and re-run playbook
[10:14] jvilla@PATH-HPWNJX3:~/projects/my-playbook $ python3.11 -m pip install cffi
Defaulting to user installation because normal site-packages is not writeable
Collecting cffi
Downloading cffi-1.17.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (467 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 467.2/467.2 KB 4.5 MB/s eta 0:00:00
Collecting pycparser
Downloading pycparser-2.22-py3-none-any.whl (117 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 KB 26.9 MB/s eta 0:00:00
Installing collected packages: pycparser, cffi
Successfully installed cffi-1.17.1 pycparser-2.22
...
...
...
...
[10:14] jvilla@PATH-HPWNJX3:~/projects/my-playbook $ ansible-playbook -vvv generate_vpn_certs.yml
ansible-playbook [core 2.17.4]
config file = /home/jvilla/projects/my-playbook/ansible.cfg
configured module search path = ['/home/jvilla/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/jvilla/.local/pipx/venvs/ansible-core/lib/python3.10/site-packages/ansible
ansible collection location = /home/jvilla/.ansible/collections:/usr/share/ansible/collections
executable location = /home/jvilla/.local/bin/ansible-playbook
python version = 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] (/home/jvilla/.local/pipx/venvs/ansible-core/bin/python)
jinja version = 3.1.4
libyaml = True
...
...
... lots of -vvv output and normal task output ...
...
...
TASK [vpn_config_generate : Generate an OpenSSL private key with the default values (4096 bits, RSA)] ***
task path: /home/jvilla/projects/my-playbook/roles/vpn_config_generate/tasks/main.yml:33
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: jvilla
<localhost> EXEC /bin/sh -c 'echo ~jvilla && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/jvilla/.ansible/tmp `"&& mkdir "` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826 `" && echo ansible-tmp-1727964878.9725957-12642-118435769711826="` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826 `" ) && sleep 0'
Using module file /home/jvilla/.ansible/collections/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py
<localhost> PUT /home/jvilla/.ansible/tmp/ansible-local-124581t36wg16/tmp1ziz87xo TO /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/AnsiballZ_openssl_privatekey.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/ /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3.11 /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
"changed": true,
"diff": {
"after": {
"can_parse_key": true,
"key_is_consistent": null,
"public_data": {
<<<redacted>>>
},
"public_key": "-----BEGIN PUBLIC KEY-----\n<<<redacted>>>\n-----END PUBLIC KEY-----\n",
"public_key_fingerprints": {
"sha256": <<<redacted>>>
},
"type": "RSA"
},
"before": {}
},
"filename": "./ssl_tmp/client.key",
"fingerprint": {
<<<redacted>>>
},
"invocation": {
"module_args": {
"attributes": null,
"backup": false,
"cipher": "auto",
"curve": null,
"force": false,
"format": "auto_ignore",
"format_mismatch": "regenerate",
"group": null,
"mode": "0600",
"owner": null,
"passphrase": null,
"path": "./ssl_tmp/client.key",
"regenerate": "full_idempotence",
"return_content": false,
"select_crypto_backend": "auto",
"selevel": null,
"serole": null,
"setype": null,
"seuser": null,
"size": 4096,
"state": "present",
"type": "RSA",
"unsafe_writes": false
}
},
"size": 4096,
"type": "RSA"
}
Read vars_file 'openvpn_vaulted.yml'
Read vars_file 'terraform_vars.yml'
The module only needs cryptography. cffi is a transitive dependency needed by cryptography: https://github.com/pyca/cryptography/blob/main/pyproject.toml#L52
@felixfontein Thanks for the response!
What do you think then is the "proper" course of action for me? This has already been filed as a bug with the cryptography project 7 years ago, and they said that it is explicitly configured to not list cffi as a dependency when installing cryptography from PyPi.
Is the requirement basically, "ensure you have cryptography installed on the target system, and also cffi if you installed cryptography via PyPi"?
The issue you are linking is about PyPy (another Python engine), not PyPI (the Python package index). If you install cryptography via pip (or any other proper resolver) for CPython (or in fact any other Python engine than PyPy), cffi should also be installed since cffi is a dependency of cryptography.
(Your Python from ansible --version doesn't look like PyPy.)
If you have cryptography installed but not cffi, then your environment is broken.
Gotcha lol, thanks for correcting my misunderstanding on the PyPy vs. PyPi 😄
then your environment is broken
This is a pretty standard install of the python3.11 apt package on Ubuntu 22.04 in WSL. Although, Ubuntu 22.04's default Python version (i.e. /usr/bin/python3) is Python 3.10, and so perhaps that is what was causing the issue?
Some details:
[18:04] jvilla@PATH-HPWNJX3:~ $ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
[17:04] jvilla@PATH-HPWNJX3:~ $
[17:04] jvilla@PATH-HPWNJX3:~ $
[17:04] jvilla@PATH-HPWNJX3:~ $
[17:04] jvilla@PATH-HPWNJX3:~ $ dpkg -l python3.11 python3-cryptography python3-cffi-backend
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==========================-==================-============-=======================================================================
ii python3-cffi-backend:amd64 1.15.0-1build2 amd64 Foreign Function Interface for Python 3 calling C code - runtime
ii python3-cryptography 3.4.8-1ubuntu2.2 amd64 Python library exposing cryptographic recipes and primitives (Python 3)
ii python3.11 3.11.0~rc1-1~22.04 amd64 Interactive high-level object-oriented language (version 3.11)
[17:04] jvilla@PATH-HPWNJX3:~ $
[17:04] jvilla@PATH-HPWNJX3:~ $
[17:04] jvilla@PATH-HPWNJX3:~ $ dpkg -L python3-cffi-backend
/.
/usr
/usr/lib
/usr/lib/python3
/usr/lib/python3/dist-packages
/usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so
/usr/share
/usr/share/doc
/usr/share/doc/python3-cffi-backend
/usr/share/doc/python3-cffi-backend/changelog.Debian.gz
/usr/share/doc/python3-cffi-backend/copyright
Not being quite this level of Python expert, the _cffi_backend.cpython-310-x86_64-linux-gnu.so seems somewhat suspicious to me, in that Python 3.11 maybe isn't able to dynamically load that shared library that's (maybe) only compatible with Python 3.10?
Not being quite this level of Python expert, the
_cffi_backend.cpython-310-x86_64-linux-gnu.soseems somewhat suspicious to me, in that Python 3.11 maybe isn't able to dynamically load that shared library that's (maybe) only compatible with Python 3.10?
This is likely the cause for your problem. _cffi_backend.cpython-310-x86_64-linux-gnu.so will only be used by Python 3.10, but not by Python 3.11. Since this file is specific to the CPython version, CPU architecture etc., all that information is encoded in the extension so you can have multiple versions present at the same time.
It's kind of strange that Python 3.10 and 3.11 use the same modules directory (/usr/lib/python3/dist-packages in your case). I'm not an expert on Python packaging, but I would expect there to be different directories for every Python version. (I think Fedora is one of the distributions which natively supports having different Python versions installed in parallel, but I don't know how they're doing that, and I don't use it so I cannot simply check ;) )