aws-load-balancer-controller icon indicating copy to clipboard operation
aws-load-balancer-controller copied to clipboard

Published 20 hours ago • verified publisher icon kubernetes-sigs

Ingress cannot be deleted when TooManyUniqueTargetGroupsPerLoadBalancer

Open jfly opened this issue 1 year ago • 6 comments

Describe the bug

We recently added a new ingress to our cluster, and it failed to deploy with this error:

Failed deploy model due to TooManyUniqueTargetGroupsPerLoadBalancer: You have reached the maximum number of unique target groups that you can associate with a load balancer of type 'application': [100] status code: 400, request id: ...

Steps to reproduce

Just keep adding new rules to your load balancer until you have over 100 target groups. Then try to remove the ingress that added the 101st target group. It won't delete, instead kubernetes will just hang waiting for the relevant finalizer to finish.

Expected outcome

I expect to be able to delete Ingresses even if they can't get created.

Environment

  • AWS Load Balancer controller version: 2.5.1
  • Kubernetes version: 1.28
  • Using EKS (yes/no), if so version? yes, platform version: eks.7

Additional Context:

jfly avatar Feb 07 '24 00:02 jfly

@jfly Have you tried removing the finalizer on your ingress and try to see if the controller then deletes this ingress for you? Also if possible could you please provide us the controller logs around the issue time so that we can look into improving this behaviour?

shraddhabang avatar Feb 07 '24 23:02 shraddhabang

@shraddhabang, we did not, although I assume that would have gotten rid of the ingress. We instead "solved" this by removing some target groups from the load balancer so the controller could finish creating the ingress and then go on to remove it.

I don't have logs, sorry. I am pretty sure this is straightforward to reproduce, though.

jfly avatar Feb 07 '24 23:02 jfly

Same issue here. Deleting the finalizer from ingress deleted the ingress, however, restart of aws-load-balancer-controller deployment was needed to get rid of the error.

AWS Load Balancer controller version: 2.4.7 Kubernetes version: 1.27 Using EKS (yes/no), if so version? yes, platform version: eks.7

traseta avatar Apr 10 '24 15:04 traseta

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jul 09 '24 15:07 k8s-triage-robot

/remove-lifecycle stale

jfly avatar Jul 09 '24 15:07 jfly

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Oct 07 '24 16:10 k8s-triage-robot