Respect AZURE_CLIENT_ID, ANSIBLE_AZURE_AUTH_SOURCE on inventory plugin

[kingsleyadam]

trafficstars

SUMMARY

Fixes #712, set the inventory plugin to respect the ANSIBLE_AZURE_AUTH_SOURCE environment variable, and ensures AZURE_CLIENT_ID environment variable is pulled in if set.

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

plugin: azure.azcollection.azure_rm

ADDITIONAL INFORMATION

Before Change with ANSIBLE_AZURE_AUTH_SOURCE and AZURE_CLIENT_ID set

Using /opt/ansible-repo/ansible.cfg as config file
[WARNING]:  * Failed to parse /opt/ansible-repo/inventory/ops-az/azure_rm.yml with ansible_collections.azure.azcollection.plugins.inventory.azure_rm plugin: Failed to get credentials. Either pass as parameters, set environment
variables, define a profile in ~/.azure/credentials, or log in with Azure CLI (`az login`).

After Change, a complete list of the inventory using ansible-inventory command.

[kingsleyadam]

@Fred-sun, any chance someone can review this PR? I'm in need this enhancement/fix and I really don't want to fork the repo.

Thanks!

[Fred-sun]

any chance someone can review this PR? I'm in need this enhancement/fix and I really don't want to fork the repo.

Thanks!

@kingsleyadam Small change requests.

[kingsleyadam]

any chance someone can review this PR? I'm in need this enhancement/fix and I really don't want to fork the repo. Thanks!

@kingsleyadam Small change requests.

Updated, thanks for reviewing!

[xuzhang3]

Changes:

1. Fix inventory ignore the ENV configure
2. Support MSI auth using custom managed identity

[xuzhang3]

@kingsleyadam can you provider test result of the fix/enhancement? I cannot get this work with MSI authorization

[testotxt]

hi any news on this? https://github.com/ansible-collections/azure/issues/712 is still present on azure.azcollection 1.12.0

[Fred-sun]

o

@testotxt This PR still needs to be improved by contributors, so the merger cannot be promoted for the time being. Thank you very much!

[Fred-sun]

@kingsleyadam Kindly ping!

[kingsleyadam]

@Fred-sun, @xuzhang3, finally getting back to this. Sorry for the delay. Here's the testing I've done from an Azure VM with multiple Managed/User Assigned Identities.

No Fix, setting MSI

This is the code before any changes. Setting ANSIBLE_AZURE_AUTH_SOURCE=msi gets ignored, the error indicates it requires the azure cli login.

(ansible) user@stackstorm:/opt/ansible-repo# export ANSIBLE_AZURE_AUTH_SOURCE=msi
(ansible) user@stackstorm:/opt/ansible-repo# ansible-inventory -i inventory/sandbox-az --graph -vvvv
ansible-inventory 2.9.27
  config file = /opt/ansible-repo/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/stackstorm/virtualenvs/ansible/lib/python3.8/site-packages/ansible
  executable location = /opt/stackstorm/virtualenvs/ansible/bin/ansible-inventory
  python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
Using /opt/ansible-repo/ansible.cfg as config file
setting up inventory plugins
[WARNING]:  * Failed to parse /opt/ansible-repo/inventory/sandbox-az/azure_rm.yml with ansible_collections.azure.azcollection.plugins.inventory.azure_rm plugin: Failed to get credentials. Either pass as parameters, set environment variables, define a profile in
~/.azure/credentials, or log in with Azure CLI (`az login`).
  File "/opt/stackstorm/virtualenvs/ansible/lib/python3.8/site-packages/ansible/inventory/manager.py", line 280, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 218, in parse
    self._credential_setup()
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 239, in _credential_setup
    self.azure_auth = AzureRMAuth(**auth_options)
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1454, in __init__
    self.fail("Failed to get credentials. Either pass as parameters, set environment variables, "
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1562, in fail
    self._fail_impl(msg)
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1565, in _default_fail_impl
    raise AzureRMAuthException(msg)

Fix with MSI but no CLIENT ID

The below includes the recommended fix. But without setting the AZURE_CLIENT_ID environment variable. This VM has multiple identities, a system assigned and a user assigned. By default azure selects the system assigned.

In this case the system assigned identity does not have any access. The error shows an attempt to use MSI, but failed due to access.

(ansible) user@stackstorm:/opt/ansible-repo# export ANSIBLE_AZURE_AUTH_SOURCE=msi
(ansible) user@stackstorm:/opt/ansible-repo# ansible-inventory -i inventory/sandbox-az --graph -vvvv
ansible-inventory 2.9.27
  config file = /opt/ansible-repo/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/stackstorm/virtualenvs/ansible/lib/python3.8/site-packages/ansible
  executable location = /opt/stackstorm/virtualenvs/ansible/bin/ansible-inventory
  python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
Using /opt/ansible-repo/ansible.cfg as config file
setting up inventory plugins
[WARNING]:  * Failed to parse /opt/ansible-repo/inventory/sandbox-az/azure_rm.yml with ansible_collections.azure.azcollection.plugins.inventory.azure_rm plugin: Failed to get MSI token: End of paging. Please check whether your machine enabled MSI or grant access
to any subscription.
  File "/opt/stackstorm/virtualenvs/ansible/lib/python3.8/site-packages/ansible/inventory/manager.py", line 280, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 221, in parse
    self._credential_setup()
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 243, in _credential_setup
    self.azure_auth = AzureRMAuth(**auth_options)
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1458, in __init__
    self.credentials = self._get_credentials(
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1720, in _get_credentials
    return self._get_msi_credentials(subscription_id=params.get('subscription_id'), client_id=params.get('client_id'),
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1666, in _get_msi_credentials
    self.fail("Failed to get MSI token: {0}. "
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1605, in fail
    self._fail_impl(msg)
  File "/home/user/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1608, in _default_fail_impl
    raise AzureRMAuthException(msg)

Fix with MSI and CLIENT ID

The following fix tests setting both ANSIBLE_AZURE_AUTH_SOURCE and AZURE_CLIENT_ID. Shows a successful call to the inventory plugin with results.

(ansible) user@stackstorm:/opt/ansible-repo# export ANSIBLE_AZURE_AUTH_SOURCE=msi
(ansible) user@stackstorm:/opt/ansible-repo# export AZURE_CLIENT_ID=<redacted>
(ansible) user@stackstorm:/opt/ansible-repo# ansible-inventory -i inventory/sandbox-az --graph -vvvv
ansible-inventory 2.9.27
  config file = /opt/ansible-repo/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/stackstorm/virtualenvs/ansible/lib/python3.8/site-packages/ansible
  executable location = /opt/stackstorm/virtualenvs/ansible/bin/ansible-inventory
  python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
Using /opt/ansible-repo/ansible.cfg as config file
setting up inventory plugins
Parsed /opt/ansible-repo/inventory/sandbox-az/azure_rm.yml inventory source with ansible_collections.azure.azcollection.plugins.inventory.azure_rm plugin
setting up inventory plugins
Parsed /opt/ansible-repo/inventory/sandbox-az/constructed.yml inventory source with constructed plugin
@all:
  |--@apiserver:
  |  |--apiserver-0.sandbox.az.internal.example.com
  |  |--apiserver-1.sandbox.az.internal.example.com
  |--@backend:
  |  |--apiserver-0.sandbox.az.internal.example.com
  |  |--apiserver-1.sandbox.az.internal.example.com
  |  |--console-0.sandbox.az.internal.example.com
  |  |--kwc-0-0.sandbox.az.internal.example.com
  |  |--nlpserver-0.sandbox.az.internal.example.com
  |  |--replicationserver-0.sandbox.az.internal.example.com
  |  |--sputnik-0.sandbox.az.internal.example.com
  |  |--workflow-0.sandbox.az.internal.example.com
  |--@chatbot:
  |  |--chatbot-0.sandbox.az.internal.example.com
  |--@console:
  |  |--console-0.sandbox.az.internal.example.com
  |--@kwc:
  |  |--kwc-0-0.sandbox.az.internal.example.com
  |--@nlpserver:
  |  |--nlpserver-0.sandbox.az.internal.example.com
  |--@rabbitmq:
  |  |--services-0.sandbox.az.internal.example.com
  |--@realm_hosts:
  |  |--apiserver-0.sandbox.az.internal.example.com
  |  |--apiserver-1.sandbox.az.internal.example.com
  |  |--chatbot-0.sandbox.az.internal.example.com
  |  |--console-0.sandbox.az.internal.example.com
  |  |--kwc-0-0.sandbox.az.internal.example.com
  |  |--nlpserver-0.sandbox.az.internal.example.com
  |  |--render-0.sandbox.az.internal.example.com
  |  |--replicationserver-0.sandbox.az.internal.example.com
  |  |--services-0.sandbox.az.internal.example.com
  |  |--services-1.sandbox.az.internal.example.com
  |  |--services-2.sandbox.az.internal.example.com
  |  |--services-3.sandbox.az.internal.example.com
  |  |--services-4.sandbox.az.internal.example.com
  |  |--services-5.sandbox.az.internal.example.com
  |  |--services-6.sandbox.az.internal.example.com
  |  |--sftp-0.sandbox.az.internal.example.com
  |  |--sputnik-0.sandbox.az.internal.example.com
  |  |--webapp-0.sandbox.az.internal.example.com
  |  |--workflow-0.sandbox.az.internal.example.com
  |--@render:
  |  |--render-0.sandbox.az.internal.example.com
  |--@replicationserver:
  |  |--replicationserver-0.sandbox.az.internal.example.com
  |--@services:
  |  |--services-0.sandbox.az.internal.example.com
  |  |--services-1.sandbox.az.internal.example.com
  |  |--services-2.sandbox.az.internal.example.com
  |  |--services-3.sandbox.az.internal.example.com
  |  |--services-4.sandbox.az.internal.example.com
  |  |--services-5.sandbox.az.internal.example.com
  |  |--services-6.sandbox.az.internal.example.com
  |--@sftp:
  |  |--sftp-0.sandbox.az.internal.example.com
  |--@sputnik:
  |  |--sputnik-0.sandbox.az.internal.example.com
  |--@ui:
  |  |--chatbot-0.sandbox.az.internal.example.com
  |  |--render-0.sandbox.az.internal.example.com
  |  |--webapp-0.sandbox.az.internal.example.com
  |--@ungrouped:
  |--@webapp:
  |  |--webapp-0.sandbox.az.internal.example.com
  |--@workflow:
  |  |--workflow-0.sandbox.az.internal.example.com

[kingsleyadam]

Friendly ping @Fred-sun

[Fred-sun]

@kingsleyadam Are you still paying attention to this PR? There is a conflict in this PR, can you help solve the conflict? I will push forward the merger of this PR as soon as possible, thank you!

[kingsleyadam]

@kingsleyadam Are you still paying attention to this PR? There is a conflict in this PR, can you help solve the conflict? I will push forward the merger of this PR as soon as possible, thank you!

Thanks for the heads up, I've merged the upstream/dev branch to mine and resolved the conflict. Should be good now.