[ISSUE] Issue with `databricks_alert_v2` resource

[ron-placer]

trafficstars

Configuration

resource "databricks_alert_v2" "recent_alert" {
  display_name = "Recent Changes Alert"
  query_text     = <<EOT
                  SELECT COUNT(*) AS recent_changes_count
                  FROM `changes`
                  WHERE timestamp >= CURRENT_DATE() - INTERVAL 1 DAY;
                  EOT
  schedule = {
    quartz_cron_expression = "0 6 * * *"
    timezone_id            = "UTC"
  }
  warehouse_id = data.databricks_sql_warehouse.sql_cluster[0].id
  evaluation = {
    comparison_operator = "LESS_THAN_OR_EQUAL"
    empty_result_state = "TRIGGERED"
    source = {
      column = {
        name = "recent_changes_count"
      }
    }
    threshold = {
      value = {
        double_value = 0
      }
    }
  }

Expected Behavior

Running apply should create the alert

Actual Behavior

Apply error:

│ Endpoint not found. Please verify that the API path is correct.

Steps to Reproduce

1. terraform apply

Terraform and provider versions

Terraform v1.10.2 on linux_amd64

• provider registry.terraform.io/databricks/databricks v1.79.1
• provider registry.terraform.io/hashicorp/google v4.47.0
• provider registry.terraform.io/hashicorp/vault v4.8.0

Is it a regression?

No

Debug Output

2025-05-19T14:40:07.553+0300 [DEBUG] provider.terraform-provider-databricks_v1.80.0: POST /oidc/v1/token
> <http.RoundTripper>
< HTTP/2.0 200 OK
< {
<  "access_token": "**REDACTED**",
<  "expires_in": 3600,
<  "scope": "all-apis",
<  "token_type": "Bearer"
< }: tf_provider_addr=[registry.terraform.io/databricks/databricks](http://registry.terraform.io/databricks/databricks) tf_rpc=ConfigureProvider @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:38 @module=databricks tf_mux_provider=tf5to6server.v5tov6Server tf_req_id=de36c971-f1e8-41e5-724d-1cca23272065 timestamp="2025-05-19T14:40:07.552+0300"
2025-05-19T14:40:07.905+0300 [DEBUG] provider.terraform-provider-databricks_v1.80.0: POST /api/2.0/alerts
> {
>  "custom_description": "Alert: No recent changes in table changes... (15 more bytes)",
>  "display_name": "Recent Changes Alert",
>  "evaluation": {
>   "comparison_operator": "LESS_THAN_OR_EQUAL",
>   "empty_result_state": "TRIGGERED",
>   "source": {},
>   "threshold": {
>    "value": {
>     "double_value": 0
>    }
>   }
>  },
>  "query_text": "         SELECT COUNT(*) AS recent_changes_count\n         FROM `... (153 more bytes)",
>  "schedule": {
>   "timezone_id": "UTC"
>  },
>  "warehouse_id": "************"
> }
< HTTP/2.0 404 Not Found
< {
<  "details": [
<   {
<    "@type": "[type.googleapis.com/google.rpc.RequestInfo](http://type.googleapis.com/google.rpc.RequestInfo)",
<    "request_id": "2ab0f688-1ebe-47cf-986c-a4d8921118ab",
<    "serving_data": ""
<   }
<  ],
<  "error_code": "ENDPOINT_NOT_FOUND",
<  "message": "Endpoint not found. Please verify that the API path is correct."
< }: tf_rpc=ConfigureProvider @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:38 @module=databricks tf_mux_provider=tf5to6server.v5tov6Server tf_provider_addr=[registry.terraform.io/databricks/databricks](http://registry.terraform.io/databricks/databricks) tf_req_id=de36c971-f1e8-41e5-724d-1cca23272065 timestamp="2025-05-19T14:40:07.904+0300"
2025-05-19T14:40:07.905+0300 [DEBUG] provider.terraform-provider-databricks_v1.80.0: non-retriable error: Endpoint not found. Please verify that the API path is correct.: tf_rpc=ConfigureProvider @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:38 @module=databricks tf_provider_addr=[registry.terraform.io/databricks/databricks](http://registry.terraform.io/databricks/databricks) tf_mux_provider=tf5to6server.v5tov6Server tf_req_id=de36c971-f1e8-41e5-724d-1cca23272065 timestamp="2025-05-19T14:40:07.904+0300"
2025-05-19T14:40:07.905+0300 [ERROR] provider.terraform-provider-databricks_v1.80.0: Response contains error diagnostic: diagnostic_summary="failed to create alert_v2" tf_proto_version=6.9 tf_provider_addr=[registry.terraform.io/databricks/databricks](http://registry.terraform.io/databricks/databricks) tf_resource_type=databricks_alert_v2 tf_rpc=ApplyResourceChange @caller=/home/runner/go/pkg/mod/github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 diagnostic_detail="Endpoint not found. Please verify that the API path is correct." diagnostic_severity=ERROR tf_req_id=4d92acf0-2e2d-84bf-bdd7-88833266e78e @module=sdk.proto timestamp="2025-05-19T14:40:07.905+0300"
2025-05-19T14:40:07.906+0300 [DEBUG] State storage *remote.State declined to persist a state snapshot
2025-05-19T14:40:07.906+0300 [ERROR] vertex "module.workflow.databricks_alert_v2.recent_alert[0]" error: failed to create alert_v2
2025-05-19T14:40:07.907+0300 [DEBUG] states/remote: state read serial is: 20; serial is: 20
2025-05-19T14:40:07.907+0300 [DEBUG] states/remote: state read lineage is: 4e8089f5-0a0a-8ace-c6ba-5d04ab7ddcc4; lineage is: 4e8089f5-0a0a-8ace-c6ba-5d04ab7ddcc4
╷

Important Factoids

Would you like to implement a fix?

Not sure if related, but the databricks cli shows similar issue compared to /api/2.0/sql/alerts:

❯ databricks --profile ****-dev api get /api/2.0/sql/alerts
{}
❯ databricks --profile ****-dev api get /api/2.0/alerts
Error: Endpoint not found. Please verify that the API path is correct.
❯ databricks --profile ****-dev api post /api/2.0/sql/alerts
Error: Invalid request: Missing alert definition
❯ databricks --profile ****-dev api post /api/2.0/alerts
Error: Endpoint not found. Please verify that the API path is correct.

and the go sdk specifies /api/2.0/alerts which is a recent change and I could not find this path in the REST API reference as well.

[doruyaar]

does your workspace has been admitted into the private preview program?

https://learn.microsoft.com/en-us/azure/databricks/security/privacy/security-profile

[atzannes]

We have the same problem and we do have this beta feature enabled in our workspace. We have a temporary work-around using CURL against their REST API but, we would really like to avoid using it.

Is there any sense of how hard this would be to solve and how soon we may expect a fix?

[mgyucht]

This API and resource are very new, and it is possible they are not accessible by all customers yet. The Terraform definition is correct, though. I've pinged the API team, hopefully they get a chance to review this.

[shlomitubul]

is there any progress with this?

[renaudhartert-db]

Hi @shlomitubul, I did not manage to reproduce this issue internally. Would you be able to share your config and debug logs by any chance?