amazon.aws
amazon.aws copied to clipboard
ansible-collections
aws_ec2 and aws sso login
Summary
I have a ansible with aws_ec2 configured and it works fine with normal aws credential ( ~/.aws/credentials and exporting the tokens). Now my company is migrating to OKTA, so i need to do aws sso login to get permissions for aws. aws cli is working fine after the login, but ansible fails, aws_ec2 report no credentials found
setting up inventory plugins
redirecting (type: inventory) ansible.builtin.aws_ec2 to amazon.aws.aws_ec2
Loading collection amazon.aws from /home/dleite/.ansible/collections/ansible_collections/amazon/aws
redirecting (type: inventory) ansible.builtin.aws_ec2 to amazon.aws.aws_ec2
[WARNING]: * Failed to parse
/home/dleite/git/ansible/hosts.staging/aws_ec2.yml with
ansible_collections.amazon.aws.plugins.inventory.aws_ec2 plugin: Unable to
locate credentials
File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/home/dleite/.ansible/collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 783, in parse
results = self._query(regions, include_filters, exclude_filters, strict_permissions)
File "/home/dleite/.ansible/collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 626, in _query
for i in self._get_instances_by_region(
File "/home/dleite/.ansible/collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 503, in _get_instances_by_region
for connection, region in self._boto3_conn(regions):
File "/home/dleite/.ansible/collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 480, in _boto3_conn
assumed_credentials = self._boto3_assume_role(credentials, region)
File "/home/dleite/.ansible/collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 438, in _boto3_assume_role
sts_session = sts_connection.assume_role(RoleArn=iam_role_arn, RoleSessionName='ansible_aws_ec2_dynamic_inventory')
File "/usr/lib/python3/dist-packages/botocore/client.py", line 316, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/lib/python3/dist-packages/botocore/client.py", line 621, in _make_api_call
http, parsed_response = self._make_request(
File "/usr/lib/python3/dist-packages/botocore/client.py", line 641, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 132, in _send_request
request = self.create_request(request_dict, operation_model)
File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 115, in create_request
self._event_emitter.emit(event_name, request=request,
File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/usr/lib/python3/dist-packages/botocore/signers.py", line 90, in handler
return self.sign(operation_name, request)
File "/usr/lib/python3/dist-packages/botocore/signers.py", line 160, in sign
auth.add_auth(request)
File "/usr/lib/python3/dist-packages/botocore/auth.py", line 357, in add_auth
raise NoCredentialsError
[WARNING]: * Failed to parse
/home/dleite/git/ansible/hosts.staging/aws_ec2.yml with ini
I tried exporting AWS_PROFILE, setting up in aws_ec2.yml the boto_profile and nothing work. ~/.aws/config is correct, generated by aws sso configure sso
I suspect that the code is trying to check for credentials and not allowing boto3 to run and use this new feature
Issue Type
Bug Report
Component Name
aws_ec2
Ansible Version
$ ansible --version
ansible [core 2.12.2]
config file = /home/dleite/git/ansible/ansible.cfg
configured module search path = ['/home/dleite/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
ansible collection location = /home/dleite/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible
python version = 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
jinja version = 2.10.1
libyaml = True
It also failed with ansible 2.9 by the way
Collection Versions
$ ansible-galaxy collection list
[DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing names to new standard, use callbacks_enabled instead. This feature will be removed from ansible-core in version
2.15. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
# /usr/lib/python3/dist-packages/ansible_collections
Collection Version
----------------------------- -------
amazon.aws 2.1.0
ansible.netcommon 2.5.0
ansible.posix 1.3.0
ansible.utils 2.4.3
ansible.windows 1.9.0
arista.eos 3.1.0
awx.awx 19.4.0
azure.azcollection 1.11.0
check_point.mgmt 2.2.2
chocolatey.chocolatey 1.1.0
cisco.aci 2.1.0
cisco.asa 2.1.0
cisco.intersight 1.0.18
cisco.ios 2.6.0
cisco.iosxr 2.6.0
cisco.ise 1.2.1
cisco.meraki 2.6.0
cisco.mso 1.3.0
cisco.nso 1.0.3
cisco.nxos 2.8.2
cisco.ucs 1.6.0
cloud.common 2.1.0
cloudscale_ch.cloud 2.2.0
community.aws 2.2.0
community.azure 1.1.0
community.ciscosmb 1.0.4
community.crypto 2.2.0
community.digitalocean 1.15.0
community.dns 2.0.6
community.docker 2.1.1
community.fortios 1.0.0
community.general 4.4.0
community.google 1.0.0
community.grafana 1.3.0
community.hashi_vault 2.2.0
community.hrobot 1.2.2
community.kubernetes 2.0.1
community.kubevirt 1.0.0
community.libvirt 1.0.2
community.mongodb 1.3.2
community.mysql 2.3.3
community.network 3.0.0
community.okd 2.1.0
community.postgresql 1.6.1
community.proxysql 1.3.1
community.rabbitmq 1.1.0
community.routeros 2.0.0
community.skydive 1.0.0
community.sops 1.2.0
community.vmware 1.17.1
community.windows 1.9.0
community.zabbix 1.5.1
containers.podman 1.9.1
cyberark.conjur 1.1.0
cyberark.pas 1.0.13
dellemc.enterprise_sonic 1.1.0
dellemc.openmanage 4.4.0
dellemc.os10 1.1.1
dellemc.os6 1.0.7
dellemc.os9 1.0.4
f5networks.f5_modules 1.14.0
fortinet.fortimanager 2.1.4
fortinet.fortios 2.1.3
frr.frr 1.0.3
gluster.gluster 1.0.2
google.cloud 1.0.2
hetzner.hcloud 1.6.0
hpe.nimble 1.1.4
ibm.qradar 1.0.3
infinidat.infinibox 1.3.3
infoblox.nios_modules 1.2.1
inspur.sm 1.3.0
junipernetworks.junos 2.8.0
kubernetes.core 2.2.3
mellanox.onyx 1.0.0
netapp.aws 21.7.0
netapp.azure 21.10.0
netapp.cloudmanager 21.13.0
netapp.elementsw 21.7.0
netapp.ontap 21.15.1
netapp.storagegrid 21.9.0
netapp.um_info 21.8.0
netapp_eseries.santricity 1.2.13
netbox.netbox 3.5.1
ngine_io.cloudstack 2.2.2
ngine_io.exoscale 1.0.0
ngine_io.vultr 1.1.0
openstack.cloud 1.6.0
openvswitch.openvswitch 2.1.0
ovirt.ovirt 1.6.6
purestorage.flasharray 1.12.1
purestorage.flashblade 1.9.0
sensu.sensu_go 1.13.0
servicenow.servicenow 1.0.6
splunk.es 1.0.2
t_systems_mms.icinga_director 1.27.0
theforeman.foreman 2.2.0
vyos.vyos 2.6.0
wti.remote 1.0.3
# /home/dleite/.ansible/collections/ansible_collections
Collection Version
--------------------- -------
amazon.aws 3.1.1
community.general 3.8.0
community.hashi_vault 1.2.0
AWS SDK versions
$ $ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /usr/lib/python3/dist-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.9.253
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: UNKNOWN
License: Apache License 2.0
Location: /usr/lib/python3/dist-packages
Requires:
Required-by:
---
Name: botocore
Version: 1.16.19
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: UNKNOWN
License: Apache License 2.0
Location: /usr/lib/python3/dist-packages
Requires:
Required-by:
Configuration
$ ansible-config dump --only-changed
[DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing names to new standard, use callbacks_enabled instead. This feature will be removed from ansible-core in version
2.15. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
CALLBACKS_ENABLED(/home/dleite/git/ansible/ansible.cfg) = ['timer']
DEFAULT_CALLBACK_PLUGIN_PATH(/home/dleite/git/ansible/ansible.cfg) = ['/home/dleite/git/ansible/extras/plugins/callback']
DEFAULT_FORKS(/home/dleite/git/ansible/ansible.cfg) = 10
DEFAULT_GATHERING(/home/dleite/git/ansible/ansible.cfg) = smart
DEFAULT_HASH_BEHAVIOUR(/home/dleite/git/ansible/ansible.cfg) = merge
DEFAULT_HOST_LIST(/home/dleite/git/ansible/ansible.cfg) = ['/home/dleite/git/ansible/hosts.test']
DEFAULT_LOOKUP_PLUGIN_PATH(/home/dleite/git/ansible/ansible.cfg) = ['/home/dleite/git/ansible/extras/plugins/lookup']
DEFAULT_MANAGED_STR(/home/dleite/git/ansible/ansible.cfg) = This file is managed by Ansible.%n
template: {file}
date: %Y-%m-%d %H:%M
user: {uid}
host: {host}
DEFAULT_ROLES_PATH(/home/dleite/git/ansible/ansible.cfg) = ['/home/dleite/git/ansible/roles/external', '/home/dleite/git/ansible/roles/internal']
DEFAULT_STDOUT_CALLBACK(/home/dleite/git/ansible/ansible.cfg) = debug
DISPLAY_SKIPPED_HOSTS(/home/dleite/git/ansible/ansible.cfg) = False
HOST_KEY_CHECKING(/home/dleite/git/ansible/ansible.cfg) = False
INTERPRETER_PYTHON(/home/dleite/git/ansible/ansible.cfg) = /usr/bin/python
INVENTORY_ENABLED(/home/dleite/git/ansible/ansible.cfg) = ['aws_ec2', 'ini', 'yaml']
MAX_FILE_SIZE_FOR_DIFF(/home/dleite/git/ansible/ansible.cfg) = 327680
RETRY_FILES_ENABLED(/home/dleite/git/ansible/ansible.cfg) = False
OS / Environment
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal
Steps to Reproduce
cat host.staging/aws_ec2.yml
plugin: aws_ec2
use_contrib_script_compatible_sanitization: true
boto_profile: aws_staging
regions:
- eu-central-1
filters:
instance-state-name : running
$ export AWS_PROFILE=aws_staging
$ ansible-inventory -i hosts.staging/ --list
Expected Results
aws sso login is working, aws s3 ls works fine, ansible aws_ec2 plugin should be able to use the same feature and list the hosts
Actual Results
[DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing names to new standard, use callbacks_enabled instead. This feature will be removed from ansible-core in version 2.15. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
[WARNING]: * Failed to parse /home/dleite/git/ansible/hosts.staging/aws_ec2.yml with ansible_collections.amazon.aws.plugins.inventory.aws_ec2 plugin: Unable to locate credentials
[WARNING]: * Failed to parse /home/dleite/git/ansible/hosts.staging/aws_ec2.yml with ini plugin: Invalid host pattern '---' supplied, '---' is normally a sign this is a YAML file.
[WARNING]: * Failed to parse /home/dleite/git/ansible/hosts.staging/aws_ec2.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory
[WARNING]: Unable to parse /home/dleite/git/ansible/hosts.staging/aws_ec2.yml as an inventory source
(...other inventory list---)
Code of Conduct
- [X] I agree to follow the Ansible Code of Conduct
Files identified in the description:
- [
plugins/inventory/aws_ec2.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/inventory/aws_ec2.py)
If these files are inaccurate, please update the component name section of the description or use the !component bot command.
i found in the net this reference for the same problem: https://www.reddit.com/r/ansible/comments/qql86z/how_can_i_use_aws_sso_with_ansible_aws_ec2/ Probably the mujahidk is not really using aws sso login and so just exporting the profile with normal tokens
@danielmotaleite Thank you for raising this. I don't have much experience with aws_ec2, but it seems to me that it does not support SSO. I'll wait for @jillr's confirmation to be sure and further clarification.
If not supported, please upgrade this issue to a feature request, we use OKTA for aws authentication, so this is required... right now, as workaround, we have to login to okta, choose the aws, profile, re-authenticate and copy the AWS_* tokens ... it works, but lot more work and annoying
@danielmotaleite
The problem is that SFAIK none of the maintainers are using AWS's SSO offering, so we can't even be sure why it's not working, or test any fixes. I, for example, have direct SAML integration[1] in place, instead of using the additional AWS SSO layer. I would have expected that authentication using the profile should have worked, but since I've not used AWS SSO I don't know what differences there might be between the standard integration and AWS SSO.
What I do find odd is where the failure is occurring, it seems to imply that the iam_role_arn parameter has been set, which doesn't match with the reproducer example you've provided.
If anyone with AWS SSO experience/access is able to produce a patch, we'll happily review the patch and can try and get it merged if it doesn't break existing functionality.
[1] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_saml.html which predates the full SSO offering by a couple of years.
@danielmotaleite From what I can tell, boto3 support for AWS SSO was not added until 1.14.0. What happens if you upgrade your boto3 version?
@gravesm that is the problem, i upgraded the boto3 to the latest version and aws sso login works with aws_ec2 without issues
I didn't noticed that i was below the boto3 recommended version. Maybe aws_ec2 should warn about a unsupported boto3 version, that would trigger a fix by the user ?
You can reuse this to add this minimum version check, if not, i think this can be closed. Thanks for the help!
@tremble Is it worth adding a note for sso concerning boto3 version? The only branch requiring boto3 >= 1.9 is stable-1.5 at the moment.
@alinabuzachis It's documented in https://github.com/ansible-collections/amazon.aws#aws-sdk-version-compatibility since the 2.0 release (where we required boto3 >= 1.15.0)
That said, it looks like I didn't update the inventory plugin docs to require a specific version of botocore/boto3. I think we should update this, and I wouldn't consider this a 'breaking' change since it's only documentation and we already say that the collection as a whole requires the botocore/boto3 versions.
I also think @danielmotaleite's suggestion that the remaining (non-module) plugins should also spit out warnings is a good idea, in combination with updating the inventory plugin docs.
As a bare minimum solution #819
I agree that we should also emit a warning where the minimum requirements aren't met so I'm going to leave this open for now.
This can actually happen to this day, if you're authed in a gov cloud region, and try to get a list from a commercial region. It seems like a warning should be thrown/regions disabled, when trying to get a list of contents for a region you don't have access to?
Grab the region they authed with, if gov cloud, disable all commercial. I'm guessing govcloud is disabled by default, and it's not an issue in the other direction.
my_session = boto3.session.Session()
my_region = my_session.region_name
