[ISSUE] Issue with `databricks_access_control_rule_set` resource cannot destory control_rule_set if service principal was manually deleted

[kinger68]

We created a Service Principal with an associated control_rule_set for the servicePrincipal/user and servicePrincipal/manager. If we issue a terraform destroy with the expected state intact, the provider destroys the resources. However, if the SP was manually deleted, when attempting the same terraform destroy, the destroy errors that the SP for the rule set cannot be found while refreshing the state and fails. It is expected that the control_rule_set should be able to be cleaned up even in such a case.

Configuration

variable "workspace" {
  description = "Workspace this SP is being added to"
  type = string
  default = "your workspace"
}

variable "databricks_account_id" {
  description = "Workspace this SP is being added to"
  type = string
  default = "your account number"
}


resource "databricks_service_principal" "test_breaking_sp" {
  display_name         = "test_breaking-sp"
  allow_cluster_create = true
  workspace_access     = true
}

data "databricks_group" "admin" {
  display_name = "${var.workspace}-admin"
}

data "databricks_group" "dataops" {
  display_name = "${var.workspace}-dataops"
}


resource "databricks_access_control_rule_set" "test_breaking_sp_access_control" {
  name = "accounts/${var.databricks_account_id}/servicePrincipals/${databricks_service_principal.test_breaking_sp.application_id}/ruleSets/default"

  grant_rules {
    principals = [data.databricks_group.dataops.acl_principal_id, data.databricks_group.admin.acl_principal_id]
    role       = "roles/servicePrincipal.user"
  }

  grant_rules {
    principals = [data.databricks_group.admin.acl_principal_id]
    role       = "roles/servicePrincipal.manager"
  }
}

Expected Behavior

When executing the terraform destroy with the service principal deleted outside of terraform, the control_rule_set resource should be destroyed and the process should continue successfully.

Actual Behavior

The terraform destroy errors and exits when refreshing the state

databricks_service_principal.test_breaking_sp: Refreshing state... [id=6313326341585941] databricks_access_control_rule_set.test_breaking_sp_access_control: Refreshing state... [id=accounts/e406bdba-d775-47e9-8aa1-960fc8365c83/servicePrincipals/ddc66c0f-21f0-448d-9ca9-aed8b5c03d5c/ruleSets/default] ╷ │ Error: cannot read access control rule set: ServicePrincipal ddc66c0f-21f0-448d-9ca9-aed8b5c03d5c not found │ │ with databricks_access_control_rule_set.test_breaking_sp_access_control, │ on sp.tf line 40, in resource "databricks_access_control_rule_set" "test_breaking_sp_access_control": │ 40: resource "databricks_access_control_rule_set" "test_breaking_sp_access_control" {

Steps to Reproduce

Terraform and provider versions

required_providers { databricks = { source = "databricks/databricks" version = "1.24.1" } aws = { source = "hashicorp/aws" version = "4.51.0" } }

terraform version 1.7.1

Is it a regression?

No

Debug Output

2024-01-25T08:29:42.715-0500 [ERROR] provider.terraform-provider-databricks_v1.24.1: Response contains error diagnostic: diagnostic_summary="cannot read access control rule set: ServicePrincipal 3820b63f-9baa-44e4-b543-b5461b89c136 not found" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks tf_resource_type=databricks_access_control_rule_set tf_rpc=ReadResource @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_severity=ERROR tf_req_id=18bd378d-56c0-931c-0002-ea9413afd7f2 diagnostic_detail= timestamp=2024-01-25T08:29:42.715-0500 2024-01-25T08:29:42.716-0500 [ERROR] vertex "databricks_access_control_rule_set.test_breaking_sp_access_control" error: cannot read access control rule set: ServicePrincipal 3820b63f-9baa-44e4-b543-b5461b89c136 not found 2024-01-25T08:29:42.716-0500 [ERROR] vertex "databricks_access_control_rule_set.test_breaking_sp_access_control" error: cannot read access control rule set: ServicePrincipal 3820b63f-9baa-44e4-b543-b5461b89c136 not found 2024-01-25T08:29:42.716-0500 [ERROR] vertex "databricks_access_control_rule_set.test_breaking_sp_access_control (expand)" error: cannot read access control rule set: ServicePrincipal 3820b63f-9baa-44e4-b543-b5461b89c136 not found

Error: cannot read access control rule set: ServicePrincipal 3820b63f-9baa-44e4-b543-b5461b89c136 not found

with databricks_access_control_rule_set.test_breaking_sp_access_control, on sp.tf line 35, in resource "databricks_access_control_rule_set" "test_breaking_sp_access_control": 35: resource "databricks_access_control_rule_set" "test_breaking_sp_access_control" {

2024-01-25T08:29:42.719-0500 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2024-01-25T08:29:42.721-0500 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.24.1/darwin_amd64/terraform-provider-databricks_v1.24.1 pid=64716 2024-01-25T08:29:42.721-0500 [DEBUG] provider: plugin exited

Important Factoids

No

Would you like to implement a fix?

No