ecs-exporter icon indicating copy to clipboard operation
ecs-exporter copied to clipboard

Limit the concurrency of AWS calls to avoid throttling

Open KensoDev opened this issue 6 years ago • 0 comments

This problem is probably exposed only at our scale but I think it's a fairly reasonable feature to add anyway.

We have 12 clusters, each cluster has between 6-45 services.

Since the call to DescribeService can only accept 10 items, it needs to be called many times over in order to describe all of our services.

We keep getting throttled by Amazon saying and failing both the exporter and other perhaps more important parts of our system.

Allowing a limit in the channel concurrency will make the calls block until other calls return and remove the actual throttling since the system won't be calling AWS api so many times.

KensoDev avatar Aug 23 '18 16:08 KensoDev