stackdriver_exporter icon indicating copy to clipboard operation
stackdriver_exporter copied to clipboard

continuously poll new projects based on filters

Open DanArlowski opened this issue 2 years ago • 0 comments

Problem Statement We have GCP projects that are created dynamically and need to be discovered by the stackdriver exporter, but currently that aren't, since the exporter fetches the projects only at startup.utilize.

Desired State The stackdriver exporter fetches the projects it needs to scrape (google.projects.filter arg) on some configurable interval, that's way we can auto discover new projects and scrape their metrics.

Solution Proposal

  • Utilize mutex and keep the prometheus registry & project list in order to register/unregister projects in a separate subroutine.
  • Create a continuous update for GCP projects using the filters,
  • The update mechanism can and should be editable to both disable the behavior as well as increasing/decreasing the time between polls

we already created a working POC, which just needs a bit more polishing

@SuperQ please let me know what do you think so we can start working on it.

DanArlowski avatar Mar 18 '24 14:03 DanArlowski