amazon.aws
amazon.aws copied to clipboard
ansible-collections
ec2_instance InvalidParameterCombination when calling ModifyInstanceAttribute
SUMMARY
We're starting to see CI failures for tests that use ec2_instance when creating new instances: "msg": "Could not apply change {'InstanceId': 'i-0c72055e2fe442009', 'Groups': []} to new instance.: An error occurred (InvalidParameterCombination) when calling the ModifyInstanceAttribute operation: No attributes specified.",
For example, https://app.shippable.com/github/ansible-collections/community.aws/runs/1144/23/console
Appears to be a failure with https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/ec2_instance.py#L1174 not returning any groups.
ISSUE TYPE
- Bug Report
COMPONENT NAME
ec2_instance.py
ANSIBLE VERSION
2.9, 2.10, devel
OS / ENVIRONMENT
shippable
Leaving some notes here as I'll be away for several days. Replicating CI using the ec2_eip tests:
At https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/ec2_instance.py#L1171
id_filters looks like:
[{'Name': 'vpc-id', 'Values': ['vpc-371b0555']}, {'Name': 'group-id', 'Values': ['sg-06ed7d30d22e0a670']}]
The problem here is that this vpc-id is the default VPC for us-east-1 in CI and sg-06ed7d30d22e0a670 (in this example) belongs to the test VPC, vpc-077adc0ebabb1f33b. So describe_security_groups() with that filter quite reasonably doesn't return anything in found_groups at https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/ec2_instance.py#L1370.
Which makes expected_groups None, which is a mismatch to instance_groups at 1378, sending an empty group list via changes_to_apply to modify_instance_attribute() at https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/ec2_instance.py#L1669
However if I run a simple playbook against a personal account using my actual default VPC, same failure An error occurred (InvalidParameterCombination) when calling the ModifyInstanceAttribute operation: No attributes specified. So, that all may be irrelevant/secondary to the actual problem but I have not debugged this as thoroughly yet. The ec2_eip tests haven't changed in 4 months so it feels like something might have changed in the way the filtered describe_security_groups() query is returning data perhaps? I wasn't able to find any references to api changes though.
This feels a lot like the instability we were seeing in https://github.com/ansible-collections/amazon.aws/pull/180 where filtered searches were (are) randomly returning empty results - I switched over to running the describe based on ID and it returned much more consistently.
@tremble My concern is if this was a default behaviour that worked previously and is now resulting in new behaviour for Ansible users, I didn't find any obvious change notices on the AWS side but that's not shocking.
@briantist I believe we've come to the conclusion that the fix here is to always be explicit and provide a vpc_subnet_id whenever you're not launching in the default VPC for the region. I believe from what you said on IRC you were just wanting to proactively track this issue, have you seen any issues that would not be solved by doing that?
Thanks @jillr , I think we're already always setting vpc_subnet_id in our use of ec2_instance 😅
Good to hear that it's (seemingly) not a fully breaking upstream change. Thank you for following up!
For anyone else who might come across this, from my debugging what I've been seeing is that some of the time filter based searches are returning an empty list when they shouldn't (or at least when historically they didn't). The account we use for CI often hits the API Rate Limits, so it's also possible that this is coming into play "somehow", either client side, or possibly on the Amazon side.
I've been working around this by relying less on filter based searches where possible (ec2_eni used to make a lot of gratuitous filter based searches). If I keep seeing this get worse I'll likely try to put together a reproducer that we can throw at Amazon.
