IAM Role not detached from instance profile and not deleted

[matkosoric]

We were getting errors with IAM roles while nuking accounts:

global - IAMRole - test-role-42021100407080948100000000d - time="2021-11-03T10:06:59Z" level=error msg="DeleteConflict: Cannot delete entity, must remove roles from instance profile first.\n\tstatus code: 409, request id: 2dabdfcb-2edb-4529-a046-542883fa9710" time="2021-11-03T10:06:59Z" level=error msg="DeleteConflict: Cannot delete entity, must remove roles from instance profile first.\n\tstatus code: 409, request id: 8098e1d0-6070-4188-8a10-f654ffddbe8e"

As a workaround, we listed all roles, filtered relevant ones, then listed instance profiles for each role, and finally within second loop actually detached roles. Here is gist:

[ekristen]

@matkosoric I haven't experience this and I have roles and instance profiles that I'm cleaning up on a regular basis. The tool retries any failures anyways. Are you still seeing this issue? Can you provide the output from the tool?

[mavogel]

I am experiencing the same issue

• aws-nuke version: v2.14.0
• output:

global - IAMUser - ABC-user - failed
--
986 | time="2021-12-06T10:01:15Z" level=error msg="DeleteConflict: Cannot delete entity, must delete login profile first.\n\tstatus code: 409, request id: 02f5f96e-9834-46cd-8bf0-f9a9cf584720"

[ekristen]

@mavogel does the output have a listing for the instance profile? IAMInstanceProfile when it's trying to find all the resources to remove?

[matkosoric]

@ekristen I can not reproduce it currently since my bash workaround is working properly and I do not have access to the repo any longer. But output is in fact from the tool, only formatted a bit weirdly by GitLab.

[matkosoric]

@ekristen IAMInstanceProfile is not listed in logs

[ekristen]

@matkosoric can you please post your configuration.

[ekristen]

@matkosoric Feel free to sanitize account ids and the like, but try and keep it intact as much as possible. I suspect that the configuration that you ran against 2.14.0 might be omitting the instance profile in some way.

[matkosoric]

It is something along following lines; JSON is transformed to YAML. IAMRole, IAMRolePolicy, and IAMRolePolicyAttachment listed here are not the ones causing issues, but roles and instances created via Terraform. That part of the code is unavailable.

filters = {
  IAMUser = [
    "000000"
  ]
  IAMUserAccessKey = [
    { property = UserName, type = regex, value = ".*0000.*"}
  ]
  IAMPolicy = [
    { type = regex, value = "policy/0000000"}
  ]
  IAMUserPolicyAttachment = [
    "0000 -> AdministratorAccess"
    "0000 -> 0000"
    "0000 -> AmazonS3FullAccess"
  ]
  IAMRole = [
    { type = regex, value = ".*0000.*" }
    "OrganizationAccountAccessRole"
    "0000"
  ]
  EC2InternetGateway = [
    { property =  DefaultForAz, value = "true" }
  ]
  EC2VPC = [
    { property = IsDefault, value = "true" }
  ]
  EC2DHCPOption = [
    { type = regex, value = ".*0000.*" }
  ]
  EC2Subnet = [
    { property = DefaultForAz, value = "true" }
  ]
  IAMRolePolicy = [
    "OrganizationAccountAccessRole -> AdministratorAccess"
    { type = regex, value = ".*0000.*" }
  ]
  IAMRolePolicyAttachment = [
    { type = regex, value = ".*0000.*" }
    "0000 -> AdministratorAccess" 
    "0000 -> AmazonRoute53FullAccess"
  ]
  S3Bucket = [
    {type = regex, value = ".*0000.*"} 
  ]
  EC2InternetGatewayAttachment = [
        "igw-00000000 -> vpc-0000"
  ]
  EC2RouteTable = [
    "rtb-0000"
  ]
  EC2InternetGateway = [
    "igw-0000"
  ]
}

[ekristen]

@mavogel you aren't limiting which resources the tool is run again using the resourceTypes part of the configuration? (ie including/excluding any resources?