amazon.aws
amazon.aws copied to clipboard
ansible-collections
`AccessDenied` when using AnsibleAWSModule for SSOAdmin actions
Summary
An AccessDenied failure is received when:
- An attempt is made to use the
SSOAdminAPI. (IAMand others work as expected.) - Running through Buildkite on an EC2 instance that uses
instance-rolesfor its AWS access. (Running from my workstation with SSO session credentials works.) - Using the AnsibleAWSModule. (_
boto3works, as does runningawsclifrom thecommandmodule in the same role)
For instance, this fails:
from ansible_collections.amazon.aws.plugins.module_utils.core import AnsibleAWSModule
def main():
""" sso permission set ansible module """
argument_spec = dict(
)
module = AnsibleAWSModule(
argument_spec=argument_spec,
)
client = module.client('sso-admin')
module.exit_json(msg=client.list_instances(), ansible_facts={})
if __name__ == '__main__':
main()
But this works:
client = module.client('iam')
module.exit_json(msg=client.list_users(), ansible_facts={})
And this also works:
boto3client = boto3.client('sso-admin', region_name='us-east-1')
module.exit_json(msg=boto3client.list_instances(), ansible_facts={})
The instance role in question receives sso:* permissions, as verified by running awscli sso-admin list-instances from the Ansible role.
~I believe there is a bug in AnsibleAWSModule or its dependencies.~ https://github.com/ansible-collections/amazon.aws/issues/537#issuecomment-947797224
Issue Type
Bug Report
Component Name
core.py
Ansible Version
$ ansible --version
ansible [core 2.11.6]
config file = /var/lib/buildkite-agent/builds/management-px-kube-1/placeexchange/px-platform-users-iam/ansible.cfg
configured module search path = ['/var/lib/buildkite-agent/builds/management-px-kube-1/placeexchange/px-platform-users-iam/programmatic-roles/library']
ansible python module location = /var/lib/buildkite-agent/.local/lib/python3.8/site-packages/ansible
ansible collection location = /var/lib/buildkite-agent/.ansible/collections:/usr/share/ansible/collections
executable location = /var/lib/buildkite-agent/.local/bin/ansible
python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
jinja version = 2.10.1
libyaml = True
Collection Versions
# /var/lib/buildkite-agent/.local/lib/python3.8/site-packages/ansible_collections
Collection Version
----------------------------- -------
amazon.aws 1.5.1
ansible.netcommon 2.4.0
ansible.posix 1.3.0
ansible.utils 2.4.0
ansible.windows 1.7.2
arista.eos 2.2.0
awx.awx 19.2.2
azure.azcollection 1.9.0
check_point.mgmt 2.0.0
chocolatey.chocolatey 1.1.0
cisco.aci 2.0.0
cisco.asa 2.0.3
cisco.intersight 1.0.17
cisco.ios 2.4.0
cisco.iosxr 2.4.0
cisco.meraki 2.4.2
cisco.mso 1.2.0
cisco.nso 1.0.3
cisco.nxos 2.5.1
cisco.ucs 1.6.0
cloudscale_ch.cloud 2.2.0
community.aws 1.5.0
community.azure 1.0.0
community.crypto 1.9.3
community.digitalocean 1.10.0
community.docker 1.9.1
community.fortios 1.0.0
community.general 3.7.0
community.google 1.0.0
community.grafana 1.2.3
community.hashi_vault 1.3.2
community.hrobot 1.1.1
community.kubernetes 1.2.1
community.kubevirt 1.0.0
community.libvirt 1.0.2
community.mongodb 1.3.1
community.mysql 2.1.1
community.network 3.0.0
community.okd 1.1.2
community.postgresql 1.4.0
community.proxysql 1.2.0
community.rabbitmq 1.1.0
community.routeros 1.2.0
community.skydive 1.0.0
community.sops 1.1.0
community.vmware 1.14.0
community.windows 1.6.0
community.zabbix 1.4.0
containers.podman 1.8.0
cyberark.conjur 1.1.0
cyberark.pas 1.0.7
dellemc.enterprise_sonic 1.1.0
dellemc.openmanage 3.6.0
dellemc.os10 1.1.1
dellemc.os6 1.0.7
dellemc.os9 1.0.4
f5networks.f5_modules 1.11.1
fortinet.fortimanager 2.1.3
fortinet.fortios 2.1.2
frr.frr 1.0.3
gluster.gluster 1.0.2
google.cloud 1.0.2
hetzner.hcloud 1.6.0
hpe.nimble 1.1.3
ibm.qradar 1.0.3
infinidat.infinibox 1.2.4
inspur.sm 1.3.0
junipernetworks.junos 2.5.0
kubernetes.core 1.2.1
mellanox.onyx 1.0.0
netapp.aws 21.6.0
netapp.azure 21.9.0
netapp.cloudmanager 21.10.0
netapp.elementsw 21.6.1
netapp.ontap 21.11.0
netapp.um_info 21.7.0
netapp_eseries.santricity 1.2.13
netbox.netbox 3.1.2
ngine_io.cloudstack 2.1.0
ngine_io.exoscale 1.0.0
ngine_io.vultr 1.1.0
openstack.cloud 1.5.1
openvswitch.openvswitch 2.0.0
ovirt.ovirt 1.6.3
purestorage.flasharray 1.10.0
purestorage.flashblade 1.6.0
sensu.sensu_go 1.12.0
servicenow.servicenow 1.0.6
splunk.es 1.0.2
t_systems_mms.icinga_director 1.22.0
theforeman.foreman 2.2.0
vyos.vyos 2.5.1
wti.remote 1.0.1
AWS SDK versions
$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /var/lib/buildkite-agent/.local/lib/python3.8/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.18.60
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /var/lib/buildkite-agent/.local/lib/python3.8/site-packages
Requires: s3transfer, botocore, jmespath
Required-by: snowflake-connector-python
---
Name: botocore
Version: 1.21.60
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /var/lib/buildkite-agent/.local/lib/python3.8/site-packages
Requires: python-dateutil, jmespath, urllib3
Required-by: s3transfer, boto3
Configuration
$ ansible-config dump --only-changed
OS / Environment
Linux 5.4.0-1045-aws #47-Ubuntu SMP Tue Apr 13 07:02:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Steps to Reproduce
- name: Create/update permission sets
aws_sso_permission_set:
region: "{{ region }}"
from ansible_collections.amazon.aws.plugins.module_utils.core import AnsibleAWSModule
def main():
""" sso permission set ansible module """
argument_spec = dict(
)
module = AnsibleAWSModule(
argument_spec=argument_spec,
)
client = module.client('sso-admin')
module.exit_json(msg=client.list_instances(), ansible_facts={})
if __name__ == '__main__':
main()
As far as I know this will only happen if you're using instance roles as on an EC2 instance.
Expected Results
I expect this to exit with the instance roles in the output.
Actual Results
TASK [aws_sso_permission_set : Create/update permission sets] ******************
task path: /var/lib/buildkite-agent/builds/management-px-kube-1/placeexchange/px-platform-users-iam/programmatic-roles/roles/aws_sso_permission_set/tasks/main.yml:9
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: buildkite-agent
<127.0.0.1> EXEC /bin/sh -c 'echo ~buildkite-agent && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /var/lib/buildkite-agent/.ansible/tmp `"&& mkdir "` echo /var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231 `" && echo ansible-tmp-1634587912.6014094-84598-229160143833231="` echo /var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231 `" ) && sleep 0'
Including module_utils file ansible/__init__.py
Including module_utils file ansible/module_utils/__init__.py
Including module_utils file ansible/module_utils/basic.py
Including module_utils file ansible/module_utils/_text.py
Including module_utils file ansible/module_utils/common/_collections_compat.py
Including module_utils file ansible/module_utils/common/__init__.py
Including module_utils file ansible/module_utils/common/_json_compat.py
Including module_utils file ansible/module_utils/common/_utils.py
Including module_utils file ansible/module_utils/common/arg_spec.py
Including module_utils file ansible/module_utils/common/file.py
Including module_utils file ansible/module_utils/common/parameters.py
Including module_utils file ansible/module_utils/common/collections.py
Including module_utils file ansible/module_utils/common/process.py
Including module_utils file ansible/module_utils/common/sys_info.py
Including module_utils file ansible/module_utils/common/text/converters.py
Including module_utils file ansible/module_utils/common/text/__init__.py
Including module_utils file ansible/module_utils/common/text/formatters.py
Including module_utils file ansible/module_utils/common/validation.py
Including module_utils file ansible/module_utils/common/warnings.py
Including module_utils file ansible/module_utils/compat/selectors.py
Including module_utils file ansible/module_utils/compat/__init__.py
Including module_utils file ansible/module_utils/compat/_selectors2.py
Including module_utils file ansible/module_utils/compat/selinux.py
Including module_utils file ansible/module_utils/distro/__init__.py
Including module_utils file ansible/module_utils/distro/_distro.py
Including module_utils file ansible/module_utils/errors.py
Including module_utils file ansible/module_utils/parsing/convert_bool.py
Including module_utils file ansible/module_utils/parsing/__init__.py
Including module_utils file ansible/module_utils/pycompat24.py
Including module_utils file ansible/module_utils/six/__init__.py
Including module_utils file ansible_collections/amazon/aws/plugins/module_utils/core.py
Including module_utils file ansible/module_utils/common/dict_transformations.py
Including module_utils file ansible_collections/__init__.py
Including module_utils file ansible_collections/amazon/__init__.py
Including module_utils file ansible_collections/amazon/aws/__init__.py
Including module_utils file ansible_collections/amazon/aws/plugins/__init__.py
Including module_utils file ansible_collections/amazon/aws/plugins/module_utils/__init__.py
Including module_utils file ansible_collections/amazon/aws/plugins/module_utils/ec2.py
Including module_utils file ansible/module_utils/ansible_release.py
Including module_utils file ansible_collections/amazon/aws/plugins/module_utils/cloud.py
Using module file /var/lib/buildkite-agent/builds/management-px-kube-1/placeexchange/px-platform-users-iam/programmatic-roles/roles/aws_sso_permission_set/library/aws_sso_permission_set.py
<127.0.0.1> PUT /var/lib/buildkite-agent/.ansible/tmp/ansible-local-84448wckonv6a/tmpc3omvs1v TO /var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/ /var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 /var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py", line 100, in <module>
_ansiballz_main()
File "/var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py", line 92, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py", line 40, in invoke_module
runpy.run_module(mod_name='ansible.modules.aws_sso_permission_set', init_globals=dict(_module_fqn='ansible.modules.aws_sso_permission_set', _modlib_path=modlib_path),
File "/usr/lib/python3.8/runpy.py", line 207, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/tmp/ansible_aws_sso_permission_set_payload_fx1xjnap/ansible_aws_sso_permission_set_payload.zip/ansible/modules/aws_sso_permission_set.py", line 23, in <module>
File "/tmp/ansible_aws_sso_permission_set_payload_fx1xjnap/ansible_aws_sso_permission_set_payload.zip/ansible/modules/aws_sso_permission_set.py", line 19, in main
File "/var/lib/buildkite-agent/.local/lib/python3.8/site-packages/botocore/client.py", line 388, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/lib/buildkite-agent/.local/lib/python3.8/site-packages/botocore/client.py", line 708, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.AccessDeniedException: An error occurred (AccessDeniedException) when calling the ListInstances operation:
fatal: [localhost]: FAILED! => {
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File \"/var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py\", line 100, in <module>\n _ansiballz_main()\n File \"/var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/var/lib/buildkite-agent/.ansible/tmp/ansible-tmp-1634587912.6014094-84598-229160143833231/AnsiballZ_aws_sso_permission_set.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible.modules.aws_sso_permission_set', init_globals=dict(_module_fqn='ansible.modules.aws_sso_permission_set', _modlib_path=modlib_path),\n File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_aws_sso_permission_set_payload_fx1xjnap/ansible_aws_sso_permission_set_payload.zip/ansible/modules/aws_sso_permission_set.py\", line 23, in <module>\n File \"/tmp/ansible_aws_sso_permission_set_payload_fx1xjnap/ansible_aws_sso_permission_set_payload.zip/ansible/modules/aws_sso_permission_set.py\", line 19, in main\n File \"/var/lib/buildkite-agent/.local/lib/python3.8/site-packages/botocore/client.py\", line 388, in _api_call\n return self._make_api_call(operation_name, kwargs)\n File \"/var/lib/buildkite-agent/.local/lib/python3.8/site-packages/botocore/client.py\", line 708, in _make_api_call\n raise error_class(parsed_response, operation_name)\nbotocore.errorfactory.AccessDeniedException: An error occurred (AccessDeniedException) when calling the ListInstances operation: \n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}
Code of Conduct
- [X] I agree to follow the Ansible Code of Conduct
Files identified in the description:
- [
plugins/module_utils/core.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/module_utils/core.py)
If these files are inaccurate, please update the component name section of the description or use the !component bot command.
Hi @Tyler-2,
The IAM service is one of a handful of "global" services (which don't need a region specified), as such it's not the greatest comparison.
- Are you seeing the same errors if you use the "ec2" service?
- Are you seeing the same errors if you explicitly pass the region parameter?
Note to anyone else: When you use an Instance Profile boto3 won't automatically provide a region, and AnsibleAWSModule may even end up passing region=None
That sounds really likely...
EC2 returns this which is basically an empty list since there are no EC2 in this us-west-2 region.
ok: [localhost] => {
"ansible_facts": {},
"changed": false,
"invocation": {
"module_args": {
"aws_access_key": null,
"aws_ca_bundle": null,
"aws_config": null,
"aws_secret_key": null,
"debug_botocore_endpoint_logs": false,
"ec2_url": null,
"profile": null,
"region": "us-west-2",
"security_token": null,
"validate_certs": true
}
},
"msg": {
"Reservations": [],
"ResponseMetadata": {
"HTTPHeaders": {
"cache-control": "no-cache, no-store",
"content-length": "230",
"content-type": "text/xml;charset=UTF-8",
"date": "Wed, 20 Oct 2021 13:39:32 GMT",
"server": "AmazonEC2",
"strict-transport-security": "max-age=31536000; includeSubDomains",
"x-amzn-requestid": "505d8a85-1acd-44c6-bf67-3faddae4a359"
},
"HTTPStatusCode": 200,
"RequestId": "505d8a85-1acd-44c6-bf67-3faddae4a359",
"RetryAttempts": 0
}
}
}
What's the right way to pass the region to AnsibleAWSModule? Looking at other Ansible modules I don't see where the various dicts that get passed around are ever actually handed to AnsibleAWSModule.
Ah, well it looks like somehow magically just passing the Ansible task the right region does the trick. So... is the behavior of AnsibleAWSModule differing from boto3 a bug worth noting... do docs just need to be updated... or should I close this out?
Leave it open for now. I think the bug is that region=None gets passed (to boto3) in some conditions
I'll test out a couple more things to hopefully bring more color to this. I'm not sure region=None is actually coming into play but we'll see.
Ah, I think this is much more simply:
- AWS SSO services aren't available in all regions and return access denied. https://docs.aws.amazon.com/general/latest/gr/sso.html
- AnsibleAWSModule and the boto3 module get their region information from different places. The boto3 module gets its region information the same way my call to awscli does.
If there's something undesirable, I'd say it's AnsibleAWSModule screaming for a region to be set when it probably doesn't need one:
"The aws_sso_permission_set module requires a region and none was found in configuration, environment variables or module parameters"
So it seems like AnsibleAWSModule can't get region info the same way that boto3 and the AWSCLI when run via command: do, and this behavior isn't super transparent. I'd say that's the "bug" I've encountered, which was then complicated by the AWS SSO service not being in the normal regions.
For completeness, my EC2 instances are running in us-east-1, so I guess AWS calls default to that region. But for use-case reasons my inventory region: was set to us-west-1, which was only being used by the module when AnsibleAWSModule is used, and AnsibleAWSModule insists on passing it in.
If there's something undesirable, I'd say it's AnsibleAWSModule screaming for a
regionto be set when it probably doesn't need one:
boto3 is actually what's saying that the region is required by raising the NoRegionError exception (which we catch and output something clean):
except botocore.exceptions.NoRegionError:
module.fail_json(msg="The %s module requires a region and none was found in configuration, "
"environment variables or module parameters" % module._name)
Looking at your boto3 example from the original comment, you're explicitly providing the boto3 region.
What's kinda complicated about expecting modules to automatically pick up configuration for things like region is that exactly how and where everything gets invoked affects which configuration files it'll try to read. (In Ansible parlance, modules are executed within the 'host' context, not the 'controller' context). When working with Instance Profiles this gets even more confusing because the 'credentials', and only credentials, are automatically pulled from the network while the region can be pulled from "somewhere" on disk but isn't guaranteed
Re-reviewing I'm not sure there's anything we can realistically do here. AccessDenied was caused by querying the "wrong" region, and NoRegionError was coming directly from boto3. As such I'm going to close this Issue.