terraform-provider-grafana

terraform-provider-grafana copied to clipboard

Terraform Version

• Terraform: 1.2.0
• Terraform Grafana Provider: 1.24.0
• Grafana: 8.5.7

Affected Resource(s)

Please list the resources as a list, for example:

• grafana_synthetic_monitoring_check
• grafana_synthetic_monitoring_probe

Terraform Configuration Files

resource "grafana_synthetic_monitoring_probe" "private" {
  name      = "a_name"
  latitude  = 0
  longitude = 0 
  region    = "APAC"
  public    = false
}

resource "grafana_synthetic_monitoring_check" "this" {
  job               = "AJob"
  target            = "https://google.com"
  enabled           = true
  probes            = grafana_synthetic_monitoring_probe.private.id

  settings {
    http {
      method              = "GET"
      fail_if_not_ssl     = false
      fail_if_ssl         = false
      ip_version          = "V4"
      no_follow_redirects = false
      valid_status_codes = [
        200,
      ]

      tls_config {}
    }
  }
}

Debug Output

Please provider a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

Will provide if needed

Panic Output

If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.

NONE

Expected Behavior

What should have happened?

The above Terraform is correct behavior so there is no issue with it. However if we follow this flow we will encounter the nested dependencies:

1. Change the name of the probe

2. Terraform will pick up the change and try to re-deploy the probe

3. New probe's id will be registered

4. Old probe will be deleted

5. Checks depends on probe to finish deleting and creating, but probe can't be deleted if the checks still depends on it.

   Error: probe delete request: status="403 Forbidden", msg="delete not allowed, probe <id> has checks assigned", err="delete not allowed"
   

Actual Behavior

What actually happened?

Like the above but when changing name or force probe to be re-created.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

Are there anything atypical about your accounts that we should know? For example: Running in EC2 Classic? Custom version of OpenStack? Tight ACLs?

None

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

None

I think probe_ids in grafana_synthetic_monitoring_check will need to be re-created instead of patching.

Is this fixable by adding a depends in the terraform file?

I know that when you want to create a private probe, and immediately assign checks to it, you need to add a depends from the check to the probe, otherwise Terraform will try to create the probes and the checks at the same time.

My feeling is we need to fix the Terraform provider to account for this dependency (delete the checks first / remove the probe from the check, then delete the probes), but I'm not sure if that's something that is usually done (or possible).

The API will refuse to do this (the error message) because we don't want to assume that the user is aware of what's going on (say "delete probe" and they didn't know there were checks using that probe).

We could add another entrypoint to make this simpler, the equivalent of "remove this probe from all the checks using it" or add a flag to the probe delete entrypoint to explicitly ask for this behavior.

The depends_on wont work with REPLACE probe. The reason is the checks will try to add update probe_ids from the new probe. But old probe can't be deleted because check still keeping the old probe_id. Hence I can only think of a bad implementations from the API.

Also tried with life_cycle/create_before_destroy.

I understand what you mean, but the problem is, I think, on the terraform provider side...

Thinking out loud...

What's happening is that terraform is instructing the API to remove a probe that still has checks assigned to it. To avoid this situation, the provider should try to:

1. Create the new probe
2. For all the checks that have the existing probe assigned to them: 2.1 Add the new probe to the check 2.2 Delete the old probe from the check
3. Delete the existing probe

The API would allow this. The steps grouped under 2 can be expressed as a single update operation. The important bit (for the API) is that step 3. cannot happen before step 2.2.

I'm not sure how this maps to the way Terraform works, in particular I don't think it understands the dependency between probes and checks, that's why you have to manually declare it.

I do understand the logic behind the APIs but that should not be mapped to Terraform side. API should handle this in the backend instead.

Anyway, I have to manually taint the resources and force re-create them. probe_ids from checks are purely additional jobs and labels (metadata like) so Terraform could just replace if something changed on probe_ids.

This one doesn't have a fix yet.