cloudwatch_exporter icon indicating copy to clipboard operation
cloudwatch_exporter copied to clipboard

Cache result of ListMetrics calls

Open krzysztof-magosa opened this issue 3 years ago • 3 comments

Proposal

Use case. Why is this important?

In my opinion calling ListMetrics every minute is waste of money - resources aren't created and removed so often. To avoid that caching could be implemented with configurable TTL. In our use case we generate about 6 times more ListMetrics than GetMetricData requests, and they both are charged same way, so that would be huge money saver.

How do you think the new configuration should look like?

Example:

list_metrics_cache_ttl: 5m

krzysztof-magosa avatar Jul 28 '22 12:07 krzysztof-magosa

Yes, that's a great idea!

matthiasr avatar Jul 28 '22 15:07 matthiasr

What should the default be? Preserve the current behavior, or can we settle on a reasonable default?

matthiasr avatar Jul 28 '22 15:07 matthiasr

In my opinion 5m could be safe default - if you poll CW every 1 minute, that will save 80% of ListMetrics calls without excessive time needed for exporter to see new resources.

krzysztof-magosa avatar Jul 29 '22 08:07 krzysztof-magosa

Seems it has been implemented by @mtt88 in #453. Thanks! I'll test functionality and close issue if it works fine.

krzysztof-magosa avatar Sep 19 '22 11:09 krzysztof-magosa

image

All good, TTL is set to 60 minutes on above screen. That will make huge difference on AWS bills, many thanks again.

krzysztof-magosa avatar Sep 19 '22 13:09 krzysztof-magosa