[ISSUE] Issue with `databricks_library` resource when cluster is automatically deleted

[andreamarin]

Configuration

resource "databricks_cluster" "this" {
  provider      = databricks.workspace
  cluster_name  = var.cluster_name
  # ...
}

resource "databricks_permissions" "this" {
  provider   = databricks.workspace

  cluster_id = databricks_cluster.this.cluster_id
  # ...
}

resource "databricks_library" "cluster-library" {
  provider  = databricks.workspace
  for_each = var.cluster_libraries

  cluster_id = databricks_cluster.this.cluster_id

  # ...
}

Expected Behavior

When a cluster is deleted automatically after being terminated for more than 30 days (databricks default behaviour) the next run of a terraform plan should recreate the cluster and all of the libraries attached

Actual Behavior

The plan does detect that the cluster and the permissions were deleted and need to be recreated but fails with the databricks_library resource because the old cluster no longer exists:

Error: cannot read library: Cluster xxxx does not exist

Steps to Reproduce

1. Create a cluster with the databricks_cluster resource and add libraries to it using the databricks_library resource
2. Permanently delete the cluster
3. Run a tf plan again

Terraform and provider versions

Terraform v1.7.4
+ provider registry.terraform.io/databricks/databricks v1.47.0
+ provider registry.terraform.io/hashicorp/aws v4.33.0

Is it a regression?

Yes, from <= v1.39 this worked correctly, i.e. all the resources (cluster, permissions and libraries) were recreated after they were permanently deleted

Debug Output

NA

Important Factoids

NA

Would you like to implement a fix?

No

[janher]

I can confirm this occurs with version 1.48.2 after I manually delete a cluster (that was also created manually, but had libraries added by Terraform). When I downgrade to databricks provider 1.38.0, the same script works. Terraform version is 1.4.2

So to reproduce this:

• create a cluster manually from the databricks UI
• install a library (e.g. with the snippet below)
• delete the cluster again (manually from the databrick UI)

data "databricks_clusters" "all" {
}

resource "databricks_library" "azure-identity" {
  for_each   = data.databricks_clusters.all.ids
  cluster_id = each.key
  pypi {
    package = "azure-identity==1.16.0"
  }
}

[alexeiser]

Seems like the bug reported with https://github.com/databricks/terraform-provider-databricks/issues/1737 and fixed in https://github.com/databricks/terraform-provider-databricks/pull/1745 has been re-introduced.