feat: cache last scrape results

[eli-jordan]

When having the exporter look at a metric over a large interval, it can take quite some time to fetch all the metrics from StackDriver, due to the large number of data points that need to be fetched. In our particular use case, the fetching takes 3-4mins. With such a long fetch time, its practically not possible to have prometheus not either timeout when scraping or have many time series go stale resulting in instant queries returning no results.

To address this, this PR adds a cache that hold the result of the most recent collection from StackDriver, and this is used to server the metrics endpoint. A new collection will be scheduled and cached once the active one completes.

[SuperQ]

This is an interesting idea. I'm thinking about implementing better options to reduce scrape time. For example, taking a metric filter as URL param.

This would allow you to fetch subsets of metrics while not having to run multiple copies of the exporter. Would this help reduce your scrape times?

[SuperQ]

We've implemented selective scrape params. So this should be less necessary. This change also needs a rebase.

[SuperQ]

Maybe @fbs is interested in this.

[fbs]

yes, ill give this a go :).