yet-another-aws-exporter icon indicating copy to clipboard operation
yet-another-aws-exporter copied to clipboard

[Feature]: Per-Scraper Config

Open jonwinton opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe. Right now, our config file is only good for disabling scrapers and scrapers are "dumb" and capture all the values around a single resource. For example, the vpc-info scraper grabs info about ALL VPCs rather than looking at a list of VPCs.

Most of the time, this is alright, but in cases where we want to have more control over how a scraper runs we should enable passing down config to each scraper.

Describe the solution you'd like The config file should includes a scrapers map, with each key corresponding to a scraper's ID.

scrapers:
    vpcInfo:
        ...map of config values

Then we pass in this in to the scraper at scrape function invocation time.

Describe alternatives you've considered At exporter startup time, we could could pass in the config to each scraper at registration time and store the config in memory for each invocation. This could save on yaml unmarshaling time for each invocation.

Additional context N/A

jonwinton avatar Nov 03 '21 14:11 jonwinton