[ISSUE] `databricks_external_location.storage_root` results in an inconsistent final plan

[jorgecarleitao]

Configuration

terraform {
  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = "=1.30.0"
    }
  }
}

variable "location" {
  type        = string
  description = "The location of the metastore"
}

variable "name" {
  type        = string
  description = "The primary name of the catalog"
}

variable "owner" {
  type        = string
  description = "The security group that owns each of the catalogs"
}

variable "account_name" {
  type        = string
  description = "The name of the storage account"
}

variable "environment" {
  type        = string
  description = "The name of the environment"
}

locals {
  tiers = ["bronze", "silver", "gold"]
  // hard-code the metastore since it is global in the tenant
  databricks_metastore_ids = {
    "westeurope" : "..."
    "northeurope" : "..."
  }

  access_connector_ids = {
    "westeurope" : "/subscriptions/..."
    "northeurope" : "/subscriptions/..."
  }
}

resource "databricks_storage_credential" "main" {
  name = format("%s-%s", var.name, var.environment)
  azure_managed_identity {
    access_connector_id = local.access_connector_ids[var.location]
  }
}

resource "databricks_external_location" "main" {
  for_each        = toset(local.tiers)
  name            = format("%s-%s-%s", var.name, each.value, var.environment)
  url             = format("abfss://%s@%s.dfs.core.windows.net", each.value, var.account_name)
  credential_name = databricks_storage_credential.main.id
}

resource "databricks_catalog" "main" {
  for_each       = toset(local.tiers)
  metastore_id   = local.databricks_metastore_ids[var.location]
  storage_root   = databricks_external_location.main[each.value].url
  name           = databricks_external_location.main[each.value].name
  isolation_mode = "OPEN"
  owner          = var.owner
  force_destroy  = true
}

Expected Behavior

Provider produces consistent final plan

Actual Behavior

Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for
│ databricks_catalog.main["silver"] to include new
│ values learned so far during apply, provider
│ "registry.terraform.io/databricks/databricks" produced an invalid new value
│ for .storage_root: was
│ cty.StringVal("abfss://[email protected]"), but
│ now
│ cty.StringVal("abfss://[email protected]/").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

the error seems obvious - how the url is created needs to be consistent with respect to the trailing slash.

Steps to Reproduce

terraform plan -out main.tfplan && terraform apply main.tfplan with corresponding variables

Terraform and provider versions

Terraform v1.6.3
on linux_amd64
+ provider registry.terraform.io/databricks/databricks v1.30.0
+ provider registry.terraform.io/hashicorp/azurerm v3.80.0

Is it a regression?

Don't know

Debug Output

Important Factoids

Azure account, but it is not obvious it matters

Would you like to implement a fix?

[donatasm]

Issue can be reproduced on AWS as well. As a workaround, running apply again helps and databricks_catalog is created.

[mikalai-t]

Facing the same issue on AWS with Databricks provider 1.37.0.

[marcin-sg]

Same here on 1.35.0. Funny that the same / a similar error was already fixed last year #2335 ....