`sumologic_monitor` not storing `payload_override` in remote state

[40Cakes]

Sumo provider version: 2.18.0

The resource sumologic_monitor is not storing the value of payload_override in state for notifications. https://registry.terraform.io/providers/SumoLogic/sumologic/latest/docs/resources/monitor

After a terraform apply, the notification.payload_override field always prompts that changes are detected, even though there aren't any. This is causing some issues with CI/CD pipelines constantly detecting these changes/drifts in config.

Example:

  # module.project.sumologic_monitor.gcp_monitor[0] will be updated in-place
  ~ resource "sumologic_monitor" "gcp_project_no_logs_monitor" {
        id                        = "XXXXXXXXXXXX"
        name                      = "XXXXXXXXXXXX"
        # (18 unchanged attributes hidden)

      ~ notifications {
            # (1 unchanged attribute hidden)

          ~ notification {
              + payload_override = jsonencode(
                    {
                      + AlertName        = "{{AlertName}}"
                      + AlertURL         = "{{AlertResponseURL}}"
                      + Description      = "{{Description}}"
                      + MonitorType      = "{{MonitorType}}"
                      + QueryURL         = "{{QueryURL}}"
                      + ResultsJson      = "{{ResultsJson}}"
                      + TriggerCondition = "{{TriggerCondition}}"
                      + TriggerTime      = "{{TriggerTime}}"
                      + TriggerTimeRange = "{{TriggerTimeRange}}"
                      + TriggerValue     = "{{TriggerValue}}"
                      + action           = "create"
                      + priority         = "P3"
                    }
                )
                # (4 unchanged attributes hidden)
            }
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Checking the remote state for the above resource reveals that payload_override field is empty:

...
            "notifications": [
              {
                "notification": [
                  {
                    "action_type": "NamedConnectionAction",
                    "connection_id": "XXXXXXXXXXXX",
                    "connection_type": "Webhook",
                    "message_body": "",
                    "payload_override": "",
...

Checking the monitor in the Sumo CIP console shows the payload_override as intended.

Is this a bug or intended?

[dlinsumo]

Internal JIRA SUMO-200457

[tarunk2]

@40Cakes Can you please paste your monitor definition? It works fine for me. I see that payload_override is populated in tfstate and terraform plan doesn't show any changes for this field.

[40Cakes]

Hey @tarunk2 sure thing.

FYI - I raised this via the support portal as well, and got this response: "Yes, this is an internal defect that our backend engineering lead has opened to track and fix the reported issue."

Please note that this is applied via Terragrunt, and the following vars are called from terragrunt.hcl.

  project_id                   = "gcp-project-id"
  siem_no_logs_time_range      = "6h"
  siem_no_logs_folder          = local.siem_no_logs_folder
  siem_no_logs_connection      = local.siem_no_logs_connection
  siem_no_logs_priority        = "P3"

Resource:

# No Logs monitor
resource "sumologic_monitor" "gcp_project_no_logs_monitor" {
  count = var.siem_no_logs_time_range != null ? 1 : 0

  name         = "No Logs monitor for GCP project ${var.project_id}"
  description  = "Alerts when logs from this GCP project (${var.project_id}) stop arriving in Sumo"
  type         = "MonitorsLibraryMonitor"
  parent_id    = var.siem_no_logs_folder
  is_disabled  = false
  content_type = "Monitor"
  monitor_type = "Logs"
  queries {
    row_id = "A"
    query  = "_source=\"${var.project_id}\""
  }
  trigger_conditions {
    logs_missing_data_condition {
      time_range = var.siem_no_logs_time_range
    }
  }
  notifications {
    notification {
      connection_type  = "Webhook"
      connection_id    = var.siem_no_logs_connection
      payload_override = <<EOF
{
  "AlertName": "{{AlertName}}",
  "Description": "{{Description}}",
  "action": "create",
  "AlertURL": "{{AlertResponseURL}}",
  "QueryURL": "{{QueryURL}}",
  "TriggerTime": "{{TriggerTime}}",
  "TriggerTimeRange": "{{TriggerTimeRange}}",
  "TriggerCondition": "{{TriggerCondition}}",
  "TriggerValue": "{{TriggerValue}}",
  "ResultsJson": "{{ResultsJson}}",
  "MonitorType": "{{MonitorType}}",
  "priority": "${var.siem_no_logs_priority}"
}
EOF
    }
    run_for_trigger_types = ["MissingData", "ResolvedMissingData"]
  }
}

[tarunk2]

Thanks for the details @40Cakes! I tried this resource and it works as expected for me. In the state file:

            "notifications": [
              {
                "notification": [
                  {
                    "action_type": "NamedConnectionAction",
                    "connection_id": "***************",
                    "connection_type": "Webhook",
                    "message_body": "",
                    "payload_override": "{\n  \"AlertName\": \"{{AlertName}}\",\n  \"Description\": \"{{Description}}\",\n  \"action\": \"create\",\n  \"AlertURL\": \"{{AlertResponseURL}}\",\n  \"QueryURL\": \"{{QueryURL}}\",\n  \"TriggerTime\": \"{{TriggerTime}}\",\n  \"TriggerTimeRange\": \"{{TriggerTimeRange}}\",\n  \"TriggerCondition\": \"{{TriggerCondition}}\",\n  \"TriggerValue\": \"{{TriggerValue}}\",\n  \"ResultsJson\": \"{{ResultsJson}}\",\n  \"MonitorType\": \"{{MonitorType}}\"\n}\n",
                    "recipients": null,
                    "subject": "",
                    "time_zone": ""
                  }
                ],
                "run_for_trigger_types": [
                  "MissingData",
                  "ResolvedMissingData"
                ]
              }
            ],

It might be easier to look at it over a call. I will ask support to reach out to you to set it up. thanks!

[xorxorjmp]

Hi, Is there a workaround for this? I am experiencing the same behavior with the latest version of the sumologic provider. The payload_override is empty in the remote state on a s3 backend.

Terraform: 1.5.7 Sumologic Provider: 2.27.0

Resource:

payload_override      = <<JSON
        {
          "rule.name": "{{Name}}",
          "rule.description": "{{Description}}",
          "query.url": "{{QueryURL}}",
          "query": "{{Query}}",
          "trigger.range": "{{TriggerTimeRange}}",
          "trigger.name": "{{TriggerTime}}",
          "alert.payload": "{{ResultsJson}}"
        }
        JSON

State:

"notifications": [
              {
                "notification": [
                  {
                    "action_type": "NamedConnectionAction",
                    "connection_id": "000000000003D33E",
                    "connection_type": "Webhook",
                    "message_body": "",
                    "payload_override": "",
                    "recipients": [],
                    "resolution_payload_override": "",
                    "subject": "",
                    "time_zone": ""
                  }
                ],
                "run_for_trigger_types": [
                  "Critical",
                  "ResolvedCritical"
                ]
              }
            ],

I am using a custom module with a dynamic block for the notifications:

variable "connection_notifications" {
  description = "Connection Notifications to be sent by the alert."

  type = list(object({
    connection_type       = string,
    connection_id         = string,
    payload_override      = optional(string),
    run_for_trigger_types = list(string)
  }))
}

# Connections Notifications
  dynamic "notifications" {
    for_each = var.connection_notifications
    content {
      run_for_trigger_types = notifications.value.run_for_trigger_types
      notification {
        connection_type  = notifications.value.connection_type
        connection_id    = notifications.value.connection_id
        payload_override = notifications.value.payload_override
      }
    }
  }