amazon.aws
amazon.aws copied to clipboard
ansible-collections
Prompted multiple times when using MFA based access to accounts
Summary
When using the amazon.aws inventory module to execute ansible tasks against EC2 instances grouped by tag, I am facing an issue where I am being prompted (MFA codes) once for each member of a group. The result is running a task against a group with 4 active instances, which will prompt for MFA 4 times. My expectation is that it would request MFA once, and possibly cache the credential for reuse similar to how the AWS CLI does.
Issue Type
Bug Report
Component Name
amazon.aws
Ansible Version
$ ansible --version
ansible [core 2.12.6]
config file = /Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg
configured module search path = ['/Users/aaronsamuel/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages/ansible
ansible collection location = /Users/aaronsamuel/.ansible/collections:/usr/share/ansible/collections
executable location = /Users/aaronsamuel/opt/anaconda3/bin/ansible
python version = 3.9.7 (default, Sep 16 2021, 08:50:36) [Clang 10.0.0 ]
jinja version = 2.11.3
libyaml = True
Collection Versions
# /Users/aaronsamuel/.ansible/collections/ansible_collections
Collection Version
---------- -------
amazon.aws 3.3.0
# /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages/ansible_collections
Collection Version
----------------------------- -------
amazon.aws 2.3.0
ansible.netcommon 2.6.1
ansible.posix 1.4.0
ansible.utils 2.6.1
ansible.windows 1.10.0
arista.eos 3.1.0
awx.awx 19.4.0
azure.azcollection 1.13.0
check_point.mgmt 2.3.0
chocolatey.chocolatey 1.2.0
cisco.aci 2.2.0
cisco.asa 2.1.0
cisco.dnac 6.4.0
cisco.intersight 1.0.19
cisco.ios 2.8.1
cisco.iosxr 2.9.0
cisco.ise 1.2.1
cisco.meraki 2.6.2
cisco.mso 1.4.0
cisco.nso 1.0.3
cisco.nxos 2.9.1
cisco.ucs 1.8.0
cloud.common 2.1.1
cloudscale_ch.cloud 2.2.2
community.aws 2.5.0
community.azure 1.1.0
community.ciscosmb 1.0.5
community.crypto 2.3.2
community.digitalocean 1.19.0
community.dns 2.2.0
community.docker 2.6.0
community.fortios 1.0.0
community.general 4.8.2
community.google 1.0.0
community.grafana 1.4.0
community.hashi_vault 2.5.0
community.hrobot 1.4.0
community.kubernetes 2.0.1
community.kubevirt 1.0.0
community.libvirt 1.1.0
community.mongodb 1.4.0
community.mysql 2.3.8
community.network 3.3.0
community.okd 2.2.0
community.postgresql 1.7.4
community.proxysql 1.4.0
community.rabbitmq 1.2.1
community.routeros 2.1.0
community.sap 1.0.0
community.sap_libs 1.1.0
community.skydive 1.0.0
community.sops 1.2.2
community.vmware 1.18.0
community.windows 1.10.0
community.zabbix 1.7.0
containers.podman 1.9.3
cyberark.conjur 1.1.0
cyberark.pas 1.0.14
dellemc.enterprise_sonic 1.1.1
dellemc.openmanage 4.4.0
dellemc.os10 1.1.1
dellemc.os6 1.0.7
dellemc.os9 1.0.4
f5networks.f5_modules 1.17.0
fortinet.fortimanager 2.1.5
fortinet.fortios 2.1.6
frr.frr 1.0.4
gluster.gluster 1.0.2
google.cloud 1.0.2
hetzner.hcloud 1.6.0
hpe.nimble 1.1.4
ibm.qradar 1.0.3
infinidat.infinibox 1.3.3
infoblox.nios_modules 1.2.2
inspur.sm 1.3.0
junipernetworks.junos 2.10.0
kubernetes.core 2.3.1
mellanox.onyx 1.0.0
netapp.aws 21.7.0
netapp.azure 21.10.0
netapp.cloudmanager 21.17.0
netapp.elementsw 21.7.0
netapp.ontap 21.19.1
netapp.storagegrid 21.10.0
netapp.um_info 21.8.0
netapp_eseries.santricity 1.3.0
netbox.netbox 3.7.1
ngine_io.cloudstack 2.2.4
ngine_io.exoscale 1.0.0
ngine_io.vultr 1.1.1
openstack.cloud 1.8.0
openvswitch.openvswitch 2.1.0
ovirt.ovirt 1.6.6
purestorage.flasharray 1.13.0
purestorage.flashblade 1.9.0
sensu.sensu_go 1.13.1
servicenow.servicenow 1.0.6
splunk.es 1.0.2
t_systems_mms.icinga_director 1.29.0
theforeman.foreman 2.2.0
vmware.vmware_rest 2.1.5
vyos.vyos 2.8.0
wti.remote 1.0.3
AWS SDK versions
$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.24.6
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages
Requires: s3transfer, jmespath, botocore
Required-by:
---
Name: botocore
Version: 1.27.6
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages
Requires: python-dateutil, jmespath, urllib3
Required-by: s3transfer, boto3
Configuration
$ ansible-config dump --only-changed
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ STDIN
│ Size: -
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ DEFAULT_HOST_LIST(/Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg) = ['/Users/aaronsamuel/devops/rearc/noop-etc/ansible/aws_ec2.yaml']
2 │ HOST_KEY_CHECKING(/Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg) = False
3 │ INVENTORY_ENABLED(/Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg) = ['aws_ec2']
OS / Environment
Operating System: macOS m1 chip
❯ sw_vers
ProductName: macOS
ProductVersion: 12.4
BuildVersion: 21F79
in rosetta terminal
Target OS: Amazon Linux 2
Steps to Reproduce
❯ ANSIBLE_CACHE_PLUGIN=memory ANSIBLE_INVENTORY_CACHE_PLUGIN=memory ANSIBLE_CONFIG=./ansible.cfg AWS_PROFILE=profile-name ansible service_control -a 'uptime' -u ec2-user --verbose --become --become-method=sudo
Using /Users/aaronsamuel/devops/project/etc/ansible/ansible.cfg as config file
Enter MFA code for arn:aws:iam::XXXXX:mfa/[email protected]:
Enter MFA code for arn:aws:iam::XXXX:mfa/[email protected]:
Expected Results
I expected the MFA prompting to happen once.
Actual Results
# I am prompted for MFA seemingly once for each member of the target group
❯ ANSIBLE_CACHE_PLUGIN=memory ANSIBLE_INVENTORY_CACHE_PLUGIN=memory ANSIBLE_CONFIG=./ansible.cfg AWS_PROFILE=profile-name ansible service_control -a 'uptime' -u ec2-user --verbose --become --become-method=sudo
Using /Users/aaronsamuel/devops/project/etc/ansible/ansible.cfg as config file
Enter MFA code for arn:aws:iam::XXXXX:mfa/[email protected]:
Enter MFA code for arn:aws:iam::XXXX:mfa/[email protected]:
Code of Conduct
- [X] I agree to follow the Ansible Code of Conduct
Files identified in the description: None
If these files are inaccurate, please update the component name section of the description or use the !component bot command.
@apsamuel I'm guessing you're providing the credentials using the assume-role-provider pattern https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#assume-role-provider ?
I've been doing some work to try and improve things here on #1271. The underlying problem relates to how the AWS SDK and APIs work: To query a region for EC2 Instances you have to establish and authenticate to each of the regions individually. Unfortunately, the APIs simply do not permit you to reuse the connections between regions (the WebUI uses some additional tricks not available to the CLI/SDK).
With the code in #1271 (won't be generally available until we release 6.0.0 because some of the changes are invasive) things should have improved somewhat. It's at a state where when "regions" is specified, you're now only prompted for an MFA token once per region (previously you may be prompted a couple of times). If you only specify one region, you're only prompted for your MFA once.
There is a separate feature which also helps things, "iam_role_arn". This causes the code to call "sts:AssumeRole" (performing the MFA authentication when required) and then the code has a session token which can be reused. However, to do so, you have to specify "iam_role_arn" in the inventory file, and the role has to permit re-assuming itself without MFA, for example:
Trust Policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Principal": {
"AWS": "arn:aws:iam::123456789012:role/AssumableRole"
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"Bool": {
"aws:MultiFactorAuthPresent": "true"
}
}
}
]
}
aws_ec2.yml (inventory file)
plugin: amazon.aws.aws_ec2
profile: ansible_test_assumable
assume_role_arn: 'arn:aws:iam::123456789012:role/AssumableRole'
Not much more can be done here. We have to open separate connections for each region, and unless someone specifies assuming a role then AWS will request a new token for each region.
With the Web UI they explicitly get a session token in the background which can be reused, we have a separate module to get these tokens, but we can only get them in specific circumstances, and I'm not sure it's appropriate for us to try and magically do this in the background, since we can't reuse them.