prometheus-rds-exporter icon indicating copy to clipboard operation
prometheus-rds-exporter copied to clipboard

Allow set scrape interval to avoid throttling errors

Open nhlushak opened this issue 1 year ago • 1 comments

Is your feature request related to a problem? Please describe.

After running RDS exporter in single region, containing 65 databases we encountered periodic errors in logs about being throttled:

04 January 2024 at 17:40 (UTC+2:00)	{"time":"2024-01-04T15:40:30.006779978Z","level":"INFO","msg":"get RDS metrics"}	rds-exporter
04 January 2024 at 17:40 (UTC+2:00)	{"time":"2024-01-04T15:40:21.450411631Z","level":"ERROR","msg":"can't scrape metrics: can't fetch RDS metrics: can't compute instance metrics for xxxxxxx: can't get log files size for 274885408640: can't describe db logs files for xxxxxxx: operation error RDS: DescribeDBLogFiles, exceeded maximum number of attempts, 3, https response error StatusCode: 400, RequestID: f1e80964-6a77-xxxxxx, api error Throttling: Rate exceeded"}	rds-exporter
04 January 2024 at 17:40 (UTC+2:00)	{"time":"2024-01-04T15:40:20.003623961Z","level":"INFO","msg":"get RDS metrics"}	rds-exporter
04 January 2024 at 17:40 (UTC+2:00)	{"time":"2024-01-04T15:40:10.001978348Z","level":"INFO","msg":"get RDS metrics"}

Describe the solution you'd like

We would like to see some option to set scrape interval for exporter, so we may find some sweetspot between metrics freshness and not being throttled by AWS API.

Describe alternatives you've considered

Run exporter as scheduled operation to scrape data once, give it back to scraper and die until next run.

nhlushak avatar Jan 04 '24 16:01 nhlushak