ansible-collections
azure_rm_keyvaultsecret_info returns empty secrets list
SUMMARY
The module "azure_rm_keyvaultsecret_info" returns always an empty secrets list without showing any errors.
Hint: Issue could be related to the (already closed) issue https://github.com/ansible-collections/azure/issues/1181
ISSUE TYPE
- Bug Report
COMPONENT NAME
azure_rm_keyvaultsecret_info
ANSIBLE VERSION
ansible [core 2.15.2]
config file = None
configured module search path = ['/home/agent/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/agent/venv/py311/lib/python3.11/site-packages/ansible
ansible collection location = /home/agent/.ansible/collections:/usr/share/ansible/collections
executable location = /home/agent/venv/py311/bin/ansible
python version = 3.11.4 (main, Jun 7 2023, 12:45:48) [GCC 11.3.0] (/home/agent/venv/py311/bin/python)
jinja version = 3.1.2
libyaml = True
COLLECTION VERSION
# /home/agent/.ansible/collections/ansible_collections
Collection Version
------------------ -------
azure.azcollection 1.16.0
# /home/agent/venv/py311/lib/python3.11/site-packages/ansible_collections
Collection Version
------------------ -------
azure.azcollection 1.16.0
CONFIGURATION
CONFIG_FILE() = None
OS / ENVIRONMENT
- OS: Ubuntu 22.04
- Ansible runs in virtual environment with Python 3.11.4
- Used repository for Python installation:
ppa:deadsnakes/ppa
STEPS TO REPRODUCE
The issue occurs when below module is executed. The result is also the same when authentication is performed not through environment variables but through parameters 'tenant', 'subscription_id', 'client_id' and 'secret'.
Remark: Consuming Azure key vault with Azure CLI works without issues, as expected. Therefore the issue isn't related to validity of credential data or connectivity to Azure cloud. There must be an issue with the Ansible Azure collection.
- name: Get value from key vault secret
azure.azcollection.azure_rm_keyvaultsecret_info:
auth_source: "env"
vault_uri: "https://my-personal-one.vault.azure.net"
name: "name-of-secret"
register: mod_output
EXPECTED RESULTS
After module execution the variable "mod_output.secrets" contains a filled list.
ACTUAL RESULTS
The variable "mod_output.secrets" contains an empty list.
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: agent
<localhost> EXEC /bin/sh -c 'echo ~agent && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/agent/.ansible/tmp `"&& mkdir "` echo /home/agent/.ansible/tmp/ansible-tmp-1691477429.4954422-15429-206284005858233 `" && echo ansible-tmp-1691477429.4954422-15429-206284005858233="` echo /home/agent/.ansible/tmp/ansible-tmp-1691477429.4954422-15429-206284005858233 `" ) && sleep 0'
Using module file /home/agent/.ansible/collections/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret_info.py
<localhost> PUT /home/agent/.ansible/tmp/ansible-local-154131sz8vvua/tmpbaxqcsuh TO /home/agent/.ansible/tmp/ansible-tmp-1691477429.4954422-15429-206284005858233/AnsiballZ_azure_rm_keyvaultsecret_info.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/agent/.ansible/tmp/ansible-tmp-1691477429.4954422-15429-206284005858233/ /home/agent/.ansible/tmp/ansible-tmp-1691477429.4954422-15429-206284005858233/AnsiballZ_azure_rm_keyvaultsecret_info.py && sleep 0'
<localhost> EXEC /bin/sh -c '/home/agent/venv/py311/bin/python /home/agent/.ansible/tmp/ansible-tmp-1691477429.4954422-15429-206284005858233/AnsiballZ_azure_rm_keyvaultsecret_info.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/agent/.ansible/tmp/ansible-tmp-1691477429.4954422-15429-206284005858233/ > /dev/null 2>&1 && sleep 0'
ok: [HOST_SRV -> localhost] => {
"changed": false,
"invocation": {
"module_args": {
"ad_user": null,
"adfs_authority_url": null,
"api_profile": "latest",
"auth_source": "env",
"cert_validation_mode": null,
"client_id": null,
"cloud_environment": "AzureCloud",
"log_mode": null,
"log_path": null,
"name": "name-of-secret",
"password": null,
"profile": null,
"secret": null,
"show_deleted_secret": false,
"subscription_id": null,
"tags": null,
"tenant": null,
"thumbprint": null,
"vault_uri": "https://my-personal-one.vault.azure.net",
"version": "current",
"x509_certificate_path": null
}
},
"secrets": []
}
I've installed the same Ansible version now with lower Python version 3.10.12 and it works without issues! The issue must be related somehow to Python version 3.11.4
ANSIBLE VERSION
ansible [core 2.15.2]
config file = None
configured module search path = ['/home/agent/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/agent/venv/py310/lib/python3.10/site-packages/ansible
ansible collection location = /home/agent/.ansible/collections:/usr/share/ansible/collections
executable location = /home/agent/venv/py310/bin/ansible
python version = 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] (/home/agent/venv/py310/bin/python)
jinja version = 3.1.2
libyaml = True
COLLECTION VERSION
# /home/agent/.ansible/collections/ansible_collections
Collection Version
------------------ -------
azure.azcollection 1.16.0
# /home/agent/venv/py310/lib/python3.10/site-packages/ansible_collections
Collection Version
------------------ -------
azure.azcollection 1.16.0
CONFIGURATION
CONFIG_FILE() = None
Also reproducing this error with: ansible-playbook [core 2.15.4] python version = 3.10.13 azure.azcollection:1.17.0
But ok with (other machine, slightly different versions) : ansible-playbook [core 2.14.5] python version = 3.10.12 azure.azcollection:1.18.1 # i also tried with 1.17.0 and working
Having a similar issue, but with ansible on
ansible [core 2.11.12]
python version = 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC]
jinja version = 3.0.3
azure.azcollection 1.10.0
@TiTi @wlitke Thank you for trying and feedback! Previously, only credentials set by 'az login' were supported, and later versions of v2.0.0 support other methods of obtaining credentials. But only if you have permission to manage this resource. Thank you!
@komglebissarov The azcollecitons has been upgraded to v2.0.0, there is a major change from before (v1.10.0 is too old), if you have the permission to manage keyvault resources, can you upgrade to the latest version and try again? Thank you very much?
Hi there, I have the same issue :
ok: [server.contoso.com > localhost] => (item={'changed': False, 'secrets': [], 'invocation': {'module_args': {'vault_uri': 'https://keyvault.vault.azure.net/', 'name': 'secret-password', 'auth_source': 'cli', 'subscription_id': '00000000-0000-0000-0000-adfc57b0cf61', 'cloud_environment': 'AzureCloud', 'api_profile': 'latest', 'version': 'current', 'show_deleted_secret': False, 'profile': None, 'client_id': None, 'secret': None, 'tenant': None, 'ad_user': None, 'password': None, 'cert_validation_mode': None, 'adfs_authority_url': None, 'log_mode': None, 'log_path': None, 'x509_certificate_path': None, 'thumbprint': None, 'tags': None}}, 'failed': False, 'item': {'description': 'Data2-HA', 'letter': 'F', 'size': 2048}, 'ansible_loop_var': 'item'}) => {
"msg": {
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"invocation": {
"module_args": {
"ad_user": null,
"adfs_authority_url": null,
"api_profile": "latest",
"auth_source": "cli",
"cert_validation_mode": null,
"client_id": null,
"cloud_environment": "AzureCloud",
"log_mode": null,
"log_path": null,
"name": "secret-password",
"password": null,
"profile": null,
"secret": null,
"show_deleted_secret": false,
"subscription_id": "00000000-0000-0000-0000-adfc57b0cf61",
"tags": null,
"tenant": null,
"thumbprint": null,
"vault_uri": "https://keyvault.vault.azure.net/",
"version": "current",
"x509_certificate_path": null
}
},
"item": {
"description": "Data2-HA",
"letter": "F",
"size": 2048
},
"secrets": []
}
}
If I try from my laptop it seems to work fine everytime, but from my pipeline using a service principal, it seems to fail. Sometime it work and return the password. Sometime it returns an empty list. I'm running 2.1.0 collection.
EDIT: I was initially using ansible 2.12.x , python 3.8.x I now tested with ansible 2.16.2 and python 3.10.13 and same behaviour.
Hi there,
With python version 3.9.13 and ansible version 2.14.3 I also receive an empty list:
TASK [Get secret value] ********************************************************
task path: /runner/project/secret_read.yml:14
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/runner/.ansible/tmp `"&& mkdir "` echo /home/runner/.ansible/tmp/ansible-tmp-1702483833.0280051-26-137001510373815 `" && echo ansible-tmp-1702483833.0280051-26-137001510373815="` echo /home/runner/.ansible/tmp/ansible-tmp-1702483833.0280051-26-137001510373815 `" ) && sleep 0'
Using module file /runner/requirements_collections/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret_info.py
<127.0.0.1> PUT /home/runner/.ansible/tmp/ansible-local-21hb6sskoq/tmpw6domf94 TO /home/runner/.ansible/tmp/ansible-tmp-1702483833.0280051-26-137001510373815/AnsiballZ_azure_rm_keyvaultsecret_info.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/runner/.ansible/tmp/ansible-tmp-1702483833.0280051-26-137001510373815/ /home/runner/.ansible/tmp/ansible-tmp-1702483833.0280051-26-137001510373815/AnsiballZ_azure_rm_keyvaultsecret_info.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3.9 /home/runner/.ansible/tmp/ansible-tmp-1702483833.0280051-26-137001510373815/AnsiballZ_azure_rm_keyvaultsecret_info.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/runner/.ansible/tmp/ansible-tmp-1702483833.0280051-26-137001510373815/ > /dev/null 2>&1 && sleep 0'
ok: [localhost -> 127.0.0.1] => {
"changed": false,
"invocation": {
"module_args": {
"ad_user": null,
"adfs_authority_url": null,
"api_profile": "latest",
"auth_source": "auto",
"cert_validation_mode": null,
"client_id": null,
"cloud_environment": "AzureCloud",
"log_mode": null,
"log_path": null,
"name": "XXX-cert-2",
"password": null,
"profile": null,
"secret": null,
"show_deleted_secret": false,
"subscription_id": null,
"tags": null,
"tenant": null,
"thumbprint": null,
"vault_uri": "https://XXXXXX.vault.azure.net/",
"version": "current",
"x509_certificate_path": null
}
},
"secrets": []
}
I'm using the ansible-automation-platform-23/ee-supported-rhel8 execution environment to run azure_rm_keyvaultsecret_info
EDIT: Version 2.1.0 of the azcollection was used and required pip dependencies of azcollection are installed on top of this EE...
@zzTriplezz @wlitke om your return is empty, not a scripting issue. Can you check if your key vault access policy allows you to access secret information with authorization? Thank you very much!
My test result:
"secrets": [
{
"attributes": {
"created": "2024-01-11T10:42:25+00:00",
"enabled": true,
"expires": "2030-03-04T04:05:06+00:00",
"not_before": "2000-01-02T01:02:03+00:00",
"recovery_level": "Recoverable+Purgeable",
"updated": "2024-01-11T10:42:25+00:00"
},
"sid": "https://vaulttest00fre3.vault.azure.net/secrets/testsecret02",
"tags": {
"delete": "on-exit",
"testing": "test"
},
"version": null
},
{
"attributes": {
"created": "2024-01-11T10:43:08+00:00",
"enabled": true,
"expires": "2030-03-04T04:05:06+00:00",
"not_before": "2000-01-02T01:02:03+00:00",
"recovery_level": "Recoverable+Purgeable",
"updated": "2024-01-11T10:43:08+00:00"
},
"sid": "https://vaulttest00fre3.vault.azure.net/secrets/testsecret03",
"tags": {
"delete": "on-exit",
"testing": "test"
},
"version": null
}
]
Hi @Fred-sun
Permissions and service principals are correctly set up for Azure since the playbook works flawlessly with an older EE based on the ee-29-rhel8 with azcollection version 1.14.0. Only the new EE (ee-supported-rhel8), where we have installed on top of it the required Python pip dependencies for the azcollection is causing problems. The service principal has Key Vault Secrets User permissions and I can list secrets via az keyvault secret list:
[root@zqachxxxx admin]# az keyvault secret list --vault-name zachXXXXppkey
[
{
"attributes": {
"created": "2023-09-21T08:37:06+00:00",
"enabled": true,
"expires": "2099-12-31T23:59:59+00:00",
"notBefore": null,
"recoveryLevel": "Recoverable",
"updated": "2023-09-21T08:37:06+00:00"
},
"contentType": "",
"id": "https://zachXXXXppkey.vault.azure.net/secrets/test-cert",
"managed": null,
"name": "test-hub-cert",
"tags": {
"global-app": "APP-XXXX",
"global-appserviceid": "XXX",
"global-cbp": "XXXX",
"global-dataclass": "XXXX",
"global-dcs": "n/a",
"global-env": "XXXX",
"global-opco": "XXXX",
"global-project": "n/a",
"global-techserviceid": "XXXXX"
}
},
{
"attributes": {
"created": "2023-05-26T18:20:39+00:00",
"enabled": true,
"expires": "2099-12-31T23:59:59+00:00",
"notBefore": null,
"recoveryLevel": "Recoverable",
"updated": "2023-05-26T18:20:39+00:00"
},
"contentType": "",
"id": "https://zachXXXXppkey.vault.azure.net/secrets/test-cert-key",
"managed": null,
"name": "test-cert-key",
"tags": {
"global-app": "APP-XXXX",
"global-appserviceid": "XXXX",
"global-cbp": "XXXX",
"global-dataclass": "XXX",
"global-dcs": "n/a",
"global-env": "XXXX",
"global-opco": "XXXX",
"global-project": "n/a",
"global-techserviceid": "XXXXX"
}
},
{
"attributes": {
"created": "2023-09-21T08:54:12+00:00",
"enabled": true,
"expires": "2099-12-31T23:59:59+00:00",
"notBefore": null,
"recoveryLevel": "Recoverable",
"updated": "2023-09-21T08:54:12+00:00"
},
"contentType": "",
"id": "https://zachXXXXppkey.vault.azure.net/secrets/test-2",
"managed": null,
"name": "test-2",
"tags": {
"global-app": "APP-XXX",
"global-appserviceid": "XXXX",
"global-cbp": "XXXX",
"global-dataclass": "XXXX",
"global-dcs": "n/a",
"global-env": "XXXX",
"global-opco": "XXXX",
"global-project": "n/a",
"global-techserviceid": "XXXX"
}
}
]
EDIT: Here is the corresponding log entry from the KV
@zzTriplezz @wlitke Can you install the latest version for testing? I really can't copy this problem here, okay? In addition, please provide environmental information and use cases, thank you very much!
Why does azcollection swallow the exception and debug log it (which can be hard to access) instead of properly reporting it and failing the module? I can't imagine that anyone would want their playbook to continue after such an error, especially if it's a TypeError with an unexpected kwarg as it was in our case.
@zzTriplezz we had the same problem, also with the AAP execution environment. For us the problem was that the pip install didn't install the proper packages (namely, an old msal version that didn't support a keyword argument) (due to the old pip resolver), passing --use-feature=2020-resolver on the requirements-azure install made it work.
@TiTi @wlitke @zzTriplezz Yes, this is indeed a problem, in general, if the resource does not exist, it will throw an exception and ignore the authorization problem, I will fix this! Thank you!
@Nilstrieb @TiTi @wlitke @zzTriplezz If the key you obtained exists and returns empty, follow the link L(https://learn.microsoft.com/en-us/azure/key-vault/general/assign-access-policy?tabs=azure-portal) to add the access policy (the application you are accessing) so that it can be accessed normally. Thank you!
#1489 add new function. If there is a permission issue, an exception will be thrown