eksctl icon indicating copy to clipboard operation
eksctl copied to clipboard

[Help] Node group deletion - Cannot delete entity, must detach all policies first

Open shyam-ks opened this issue 3 years ago • 5 comments

In our cloud, a policy gets automatically attached to the node group IAM role. its part of Infra security automation. Because it's not part of eksctl creation, while deleting the node group, it fails due the stack deletion policy

Cannot delete entity, must detach all policies first. (Service: AmazonIdentityManagement; Status Code: 409; Error Code: DeleteConflict; Request ID: ffbfaa5c-a0c3-4ebc-8392-ea1ad29e432e; Proxy: null)

Is there a way to force delete this role during node group deletion using eksctl even if there are additional policies attached to it?

image

shyam-ks avatar Jun 15 '22 14:06 shyam-ks

eksctl does not support force-deleting a nodegroup's IAM role if it had policies attached outside of eksctl. The recommended way is to use nodeGroup.iam.attachPolicyARNs to specify custom policies. Please note that when you use attachPolicyARNs, eksctl does not add the default node policies, so you'll have to specify them also, as in:

nodeGroups:
  - name: test
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - <custom-policy>

cPu1 avatar Jun 21 '22 09:06 cPu1

eksctl does not support force-deleting a nodegroup's IAM role if it had policies attached outside of eksctl. The recommended way is to use nodeGroup.iam.attachPolicyARNs to specify custom policies. Please note that when you use attachPolicyARNs, eksctl does not add the default node policies, so you'll have to specify them also, as in:

nodeGroups:
  - name: test
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - <custom-policy>

There are chances that the policies get added after a while its got created. in that case, eksctl fails to delete the nodegroup. is there a way to update an existing node group with a missing policy arn? Currently not able to delete the cluster itself because of it's stuck on node group.

shyam-ks avatar Jun 23 '22 14:06 shyam-ks

eksctl does not support force-deleting a nodegroup's IAM role if it had policies attached outside of eksctl. The recommended way is to use nodeGroup.iam.attachPolicyARNs to specify custom policies. Please note that when you use attachPolicyARNs, eksctl does not add the default node policies, so you'll have to specify them also, as in:

nodeGroups:
  - name: test
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - <custom-policy>

There are chances that the policies get added after a while its got created. in that case, eksctl fails to delete the nodegroup. is there a way to update an existing node group with a missing policy arn? Currently not able to delete the cluster itself because of it's stuck on node group.

That's not supported, you'll have to detach the external policies first before running eksctl delete cluster. A simple script calling DetachRolePolicy on the node role should work.

cPu1 avatar Jun 23 '22 15:06 cPu1

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Jul 24 '22 02:07 github-actions[bot]

eksctl does not support force-deleting a nodegroup's IAM role if it had policies attached outside of eksctl. The recommended way is to use nodeGroup.iam.attachPolicyARNs to specify custom policies. Please note that when you use attachPolicyARNs, eksctl does not add the default node policies, so you'll have to specify them also, as in:

nodeGroups:
  - name: test
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - <custom-policy>

There are chances that the policies get added after a while its got created. in that case, eksctl fails to delete the nodegroup. is there a way to update an existing node group with a missing policy arn? Currently not able to delete the cluster itself because of it's stuck on node group.

That's not supported, you'll have to detach the external policies first before running eksctl delete cluster. A simple script calling DetachRolePolicy on the node role should work.

Does this help? Please let us know so we can close the issue.

cPu1 avatar Jul 25 '22 07:07 cPu1

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Aug 25 '22 02:08 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Aug 30 '22 02:08 github-actions[bot]

@shyam-ks, please open a new issue if you have any more questions.

cPu1 avatar Aug 30 '22 08:08 cPu1