Error when creating "newrelic_notification_channel" Error: UNKNOWN_ERROR: An unknown error has occurred

[ayk33]

• [x] Your New Relic provider configuration (sensitive details redacted)
• [x] A list of affected resources and/or data sources
• [x] The configuration of the resources and/or data sources related to the bug report (i.e. from the list mentioned above)
• [x] Description of the current behavior (the bug)
• [x] Description of the expected behavior

We are using terragrunt but at the end of the day it shouldn't matter.

Terraform Version

Terraform v1.3.2
on darwin_arm64
+ provider registry.terraform.io/hashicorp/google v4.40.0
+ provider registry.terraform.io/newrelic/newrelic v3.4.4

Affected Resource(s)

Please list the resources as a list, for example:

• newrelic_notification_channel
• newrelic_workflow

The terraform plan fails when attempting to create the notification_channel. As the workflow needs that as a dependency I do not know if the workflow will fail to create or not. I have not tested the terraform for the workflow code yet so there might be a bug in it. I don't think that would be related to this issue though.

Terraform Configuration

provider.tf

# Generated by Terragrunt.
data "google_secret_manager_secret_version" "newrelic_connection_info_workflow_root" {
  secret = "xxxx"
}


provider "newrelic" {
    alias      = "workflow_root"
    account_id = jsondecode(data.google_secret_manager_secret_version.newrelic_connection_info_workflow_root.secret_data).account_id
    api_key    = jsondecode(data.google_secret_manager_secret_version.newrelic_connection_info_workflow_root.secret_data).api_key
    region     = jsondecode(data.google_secret_manager_secret_version.newrelic_connection_info_workflow_root.secret_data).region
}

notification_channel.tf

locals {
  channels_list = flatten([
    for key, workflow in var.workflows : [
      for channel in workflow.slack_channel_names : {
        workflow_name      = key
        slack_channel_name = channel
      }
    ]
  ])
}

resource "newrelic_notification_channel" "channels" {
  for_each       = { for val in local.channels_list : "${val.workflow_name}-${val.slack_channel_name}" => val }
  name           = "${each.value.workflow_name}-${each.value.slack_channel_name}"
  type           = "SLACK"
  destination_id = var.destination_id
  product        = "IINT" # workflows - no idea why it's called IINT
  property {
    key   = "channelId"
    value = each.value.slack_channel_name
  }
}

I tested a different recommended solution where I explicitly set the account_id and that still did not work.

workflow.tf

resource "newrelic_workflow" "workflows" {
  for_each = var.workflows
  name     = each.key
  enabled  = try(each.value.enabled, true)

  enrichments_enabled   = each.value.enrichments_enabled
  destinations_enabled  = each.value.destinations_enabled
  muting_rules_handling = each.value.muting_rules_handling

  issues_filter {
    name = each.value.issues_filter.name
    type = each.value.issues_filter.type

    dynamic "predicate" {
      for_each = { for idx, item in each.value.issues_filter.predicate : idx => item }
      content {
        attribute = predicate.value.attribute
        operator  = predicate.value.operator
        values    = predicate.value.values
      }
    }
  }

  dynamic "destination" {
    for_each = { for idx, channel in each.value.slack_channel_names : idx => channel }
    content {
      channel_id = newrelic_notification_channel.channels["${each.key}-${destination.value}"].id
    }
  }
}

Actual Behavior

When running the terraform plan I get the following error:

newrelic_notification_channel.channels["xxxxxx-alerts-dev"]: Creating...
╷
│ Warning: Applied changes may be incomplete
│
│ The plan was created with the -target option in effect, so some changes
│ requested in the configuration may have been ignored and the output values
│ may not be fully updated. Run the following command to verify that no other
│ changes are pending:
│     terraform plan
│
│ Note that the -target option is not suitable for routine use, and is
│ provided only for exceptional situations such as recovering from errors or
│ mistakes, or when Terraform specifically suggests to use it as part of an
│ error message.
╵
╷
│ Error: UNKNOWN_ERROR: An unknown error has occurred
│
│   with newrelic_notification_channel.channels["xxxxxx-alerts-dev"],
│   on notification_channel.tf line 1, in resource "newrelic_notification_channel" "channels":
│    1: resource "newrelic_notification_channel" "channels" {
│
╵
ERRO[0018] 1 error occurred:
	* exit status 1

Expected Behavior

The terraform plan should go through without error and create the notification channel

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terragrunt apply

Debug Output

Here is the generic plan for the resources - if needed I'll run terragrunt with TF_LOG=debug.

Terraform will perform the following actions:

  # newrelic_notification_channel.channels["xxxxxx-alerts-dev"] will be created
  + resource "newrelic_notification_channel" "channels" {
      + active         = true
      + destination_id = "xxxxx"
      + id             = (known after apply)
      + name           = "xxxxxx-alerts-dev"
      + product        = "IINT"
      + status         = (known after apply)
      + type           = "SLACK"

      + property {
          + key   = "channelId"
          + value = "xxxxxxs-dev"
        }
    }

  # newrelic_workflow.workflows["xxxxx-workflow"] will be created
  + resource "newrelic_workflow" "workflows" {
      + destinations_enabled  = true
      + enabled               = true
      + enrichments_enabled   = false
      + id                    = (known after apply)
      + last_run              = (known after apply)
      + muting_rules_handling = "NOTIFY_ALL_ISSUES"
      + name                  = "xxxxx-dev-workflow"
      + workflow_id           = (known after apply)

      + destination {
          + channel_id = (known after apply)
          + name       = (known after apply)
          + type       = (known after apply)
        }

      + issues_filter {
          + filter_id = (known after apply)
          + name      = "testing-filter"
          + type      = "FILTER"

          + predicate {
              + attribute = "accumulations.policyName"
              + operator  = "EXACTLY_MATCHES"
              + values    = [
                  + "xxxx",
                  + "xxxxxxx",
                ]
            }
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Important Factoids

Right now we are running on the old newrelic account structure for environments. We are going to be migrating over to the singleton environment soon. Doc for process we will follow here. I have tried creating a workflow/channel in the environment root account as well as the sub account but got the same error.

References

I think this might be related?

• https://github.com/newrelic/terraform-provider-newrelic/issues/2009

[ayk33]

Quick update: We have done the migration to singleton environment per newrelic account and the issue is still happening.

[ayk33]

I think I might have figured out the root of the issue. For key = "channelID" under property - I'm sending in a slack channel name. I'm not sure how to get the ID of a slack channel for this integration though? Is there another key i can set besides channelID? I looked through all the docs I could find and can't see any alternative keys that can be set?

I have also updated to the newest version of the provider and still see the same issue v3.5.0

[ayk33]

Got this working. For anyone who stumbles onto this in the future, the issue was what I stated in my last comment. Needed to grab channelId from slack ui and hardcode that into my terraform.

Ideally we would get some better error from the provider to point in the right direction - instead of just "unknown error".

[arhill05]

Just to bring some more awareness - I just ran into this issue as well, and the above fix was my solution too.

[kidk]

Thanks for feedback. We'll look to improve the error message or at least guide towards a solution through the docs.