aws-load-balancer-controller icon indicating copy to clipboard operation
aws-load-balancer-controller copied to clipboard

cache TargetGroup tags to reduce number of DescribeTags calls

Open kishorj opened this issue 2 years ago • 5 comments

Is your feature request related to a problem? In case of the a lot of target groups, for example 5K, on a single VPC, and the VPC is shared across multiple k8s clusters and each cluster runs the LBC, the ingress reconciler encounters ThrottlingException for the the ELBv2 DescribeTags calls.

Describe the solution you'd like Controller cache the TargetGroup tag information to reduce the number of API calls

Describe alternatives you've considered A lower client side throttle configuration is not desirable as it slows down the reconcile of the ingress/service resources. Splitting the clusters across multiple VPCs would also require changes to the routing configuration.

kishorj avatar Apr 21 '23 23:04 kishorj

/kind feature

kishorj avatar Apr 27 '23 16:04 kishorj

We're also running into this issue. Here are the top api calls made by our load balancer controller over a period of 24 hours - it seems like DescribeTags is taking 10x the volume of any other call and I believe it's causing us to run into very long sync delays:

Screenshot 2023-05-18 at 4 33 12 PM

arkadiyt avatar May 19 '23 17:05 arkadiyt

@arkadiyt, we are supporting AWS RGT API as an alternative to solve this issue since v2.5.2. You can check our release note for more detailed info on how to enable the RGT feature.

oliviassss avatar Jun 23 '23 17:06 oliviassss

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jan 23 '24 06:01 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Feb 22 '24 06:02 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-triage-robot avatar Mar 23 '24 07:03 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Mar 23 '24 07:03 k8s-ci-robot