synthetic-monitoring-app icon indicating copy to clipboard operation
synthetic-monitoring-app copied to clipboard

Published 20 hours ago • verified publisher icon grafana

Provision the plugin with datasource UIDs instead of names

Open rdubrock opened this issue 3 years ago • 3 comments

Renaming a stack in Cloud results in the SM plugin losing track of which prometheus and loki datasources it should be talking to. This happens because prom/loki datasources get renamed when the stack gets renames, but the provisioning that informs the plugin doesn't have the new names. The plugin identifies datasources by name, so things stop working. We should look into updating the plugin provisioning to use UIDs instead (if possible), it will be more stable and the SM plugin will continue to work without interruption across renames.

rdubrock avatar Jan 31 '22 22:01 rdubrock

Thanks for creating this, @rdubrock! Some background from the HG side:

  • Currently in production the grafanaName in the json data isn't changed for SM, but I'm working on changing the datasources names now as the datasources are renamed issue here. This will still require an uninstall & reinstall, though, to get SM up and running again.
  • HG names datasources as grafanacloud-<slug>-logs but keeps the UIDs consistent across all instances so that we are able to key off of those (they will look like grafanacloud-logs or grafanacloud-prom).
  • HG will provision the datasources via a yaml file, with the UID specified, so the UID will exist from the start (in cloud at least).

Also, in case it's helpful to reference, the k8s integration uses UID of the prometheus datasource in order to grab the scrape interval from that config.

stephaniehingtgen avatar Jan 31 '22 22:01 stephaniehingtgen

Adding some thoughts after a bit of looking:

  • We have to support looking up the datasources by name AND uid. Since the SM plugin can be installed anywhere we aren't guaranteed to have provisioning or the consistent cloud uids
  • We can use UIDs instead of names for generating the SM dashboards
  • We're going to need to do some sort of migration for existing tenants to move them from name => uid in cloud

rdubrock avatar Feb 03 '22 22:02 rdubrock

Since this is merged, can this issue be closed or is there missing work?

mem avatar May 18 '22 12:05 mem

We got a report that renaming stacks will break SM until the datasource is deleted, so there is still an issue here.

rdubrock avatar Apr 17 '23 15:04 rdubrock

This should be fixed with version 1.11.0

rdubrock avatar Jun 05 '23 17:06 rdubrock