High number of requests to Stackdriver API – optimization possible?

[elsbrock]

Hi,

we're using the stackdriver exporter for our project and currently use a configuration where we just specify the prefix of the metrics we would like to request (so that the exporter requests all of them).

We suspect that this resulted in a significant increase of our bill because it apparently involves a large number of requests.

We currently export pubsub.googleapis.com, cloudsql.googleapis.com and compute.googleapis.com. The number of requests per month with two exporters deployed is about 50 million (with default settings otherwise, so I assume a refresh every 5 minutes).

Could that be related? If yes, do you see a possibility to optimize this? Is it possible to request multiple metrics at once? Could the list of metrics available per prefix be cached?

Thanks.

[SuperQ]

I'm not sure how much optimization is possible, but I'm interested in doing some work on this exporter.

[jakubbujny]

I would recommend to check which jobs are scraping SD exporter by using topk(10, count by ( job)({__name__=~".*stackdriver.*"})) query - in our case it was misconfiguration causing 3 independent Prometheus instances (HA) with 3 independent jobs (misconfiguration) scraping SD exporter (every scrape meant ~350 requests to GCP API in our configuration).

[WojciechKuk]

check https://github.com/prometheus-community/stackdriver_exporter/pull/125