"grafana_alert_notification" "secure_settings" forcing replacement on every plan.

[logan-bobo]

Terraform Version

• Terraform: v1.0.9
• Terraform Grafana Provider: 1.18.0
• Grafana: 7.3.4

Affected Resource(s)

*grafana_alert_notification

Terraform Configuration Files

resource "grafana_alert_notification" "test" {
  name          = "test alerts"
  type          = "slack"
  is_default    = true

  settings = {
    username = "test"
    recipient = "#test"
  }

  secure_settings = {
    url = var.test
  }
}

Expected Behavior

The resource should not be redeployed on every plan/apply as this is static in a tfvarsfile

Actual Behavior

On every plan, we are given the same output for this resource

  # module.test.grafana_alert_notification.test will be updated in-place
  ~ resource "grafana_alert_notification" "test" {
        id                      = "2"
        name                    = "test alerts"
      ~ secure_settings         = (sensitive value)
        # (7 unchanged attributes hidden)
    }

Important Factoids

Note this is running from a custom module where we pass in the slack webhook as a var

[logan-bobo]

Note to ensure that the issue was not the var.test I was passing in I switched this to a string. This is persistent through several plans and applies.

resource "grafana_alert_notification" "test" {
  name          = "test alerts"
  type          = "slack"
  is_default    = true

  settings = {
    username = "test"
    recipient = "#test"
  }

  secure_settings = {
    url = "string"
  }
}

  # module.test.grafana_alert_notification.test will be updated in-place
  ~ resource "grafana_alert_notification" "test" {
        id                      = "2"
        name                    = "slack alerts"
      ~ secure_settings         = (sensitive value)
        # (6 unchanged attributes hidden)
    }

[logan-bobo]

My only thought around this is if you query /api/alert-notifications you are handed back

"secureFields": {
      "url": true

and in our terraform we are setting

secure_settings = {
    url = "string"
  }

Reflecting over the grafana documentation I can see the following The following sections detail the supported settings and secure settings for each alert notification type. Secure settings are stored encrypted in the database and you add them to secure_settings in the YAML file instead of settings. how does the provider check if the value of url has changed in the secure settings? Does the provider detect a change because it is being handed back true from the API and updating it with our webhook URL?

[inkel]

I tried with a similar configuration and I can confirm that right after running terraform apply without changing the resource definition at all, a new plan/apply will report that the resource has changed its secure settings:

resource "grafana_alert_notification" "issue347" {
  name       = "test alerts"
  type       = "webhook"
  is_default = true

  settings = {
    url = "http://example.com/webhook"
  }

  secure_settings = {
    username = "inkel"
  }
}

I don't think this is a bug per se, as the value never gets rewritten nor anything, but rather a security feature that gets in the middle. Let me explain.

First of anything, I ran terraform apply with the resource definition above, so that's my initial state.

Before Terraform applies any changes it needs to get the current value of the resource and compare it with whatever the resource value has in the Terraform state. In order to do this, it performs a GET request to get the value and Grafana returns something like this:

{
  "id": 35,
  "uid": "IY1W5Dxnz",
  "name": "test alerts",
  "type": "webhook",
  "isDefault": true,
  "sendReminder": false,
  "disableResolveMessage": false,
  "frequency": "",
  "created": "2022-01-24T14:58:53-03:00",
  "updated": "2022-01-24T15:06:32-03:00",
  "settings": {
    "url": "http://example.com/webhook"
  },
  "secureFields": {
    "username": true
  }
}

As you can see, it doesn't report back the secureSettings value, just that username was a secure field. So when Terraform compares this against the resource definition it finds that there is a difference between Terraform state and the Grafana state, and thus, it says the resource has changes and tries to re-apply it.

I'm going to discuss this with my fellow maintainers to see what their thoughts are.

[logan-bobo]

Thanks for the reply! Happy to provide any further information or assisting with the fix!

[conorevans]

Hey, was there any progress on this?

[julienduchesne]

grafana_alert_notification is now deprecated in favor of the new alerting features. See the Alerting category. So this issue will not be fixed