[ISSUE] Issue with `databricks_permissions` resource - unable to grant token_usage to a service principal in Azure Government Cloud

[qingsun-db]

Configuration

Copy-paste your Terraform configuration here

resource "databricks_permissions" "token_usage" {
  provider = databricks.this_workspace
  depends_on = [var.workspace, databricks_service_principal.this]
  authorization = "tokens"
  access_control {
   service_principal_name = databricks_service_principal.this.application_id
   permission_level    = "CAN_USE"
 }
}

Expected Behavior

Service Principal with the application_id should be granted with access to tokens api

Actual Behavior

Received a “RESOURCE_DOES_NOT_EXIST”, “tokens tokens does not exist” error. Debug log is enclosed below

2022-08-07T13:49:49.597-0400 [DEBUG] provider.terraform-provider-databricks_v1.2.0.exe: 404 Not Found {
  “error_code”: “RESOURCE_DOES_NOT_EXIST”,
  “message”: “tokens tokens does not exist”
}: timestamp=2022-08-07T13:49:49.597-0400
2022-08-07T13:49:49.598-0400 [ERROR] provider.terraform-provider-databricks_v1.2.0.exe: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR diagnostic_summary=“cannot create permissions: tokens tokens does not exist” tf_provider_addr=provider tf_req_id=fac59662-67e7-be9f-520a-b7c425ba950e @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/[github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go](http://github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go):56 @module=sdk.proto tf_proto_version=5.3 tf_resource_type=databricks_permissions tf_rpc=ApplyResourceChange timestamp=2022-08-07T13:49:49.598-0400
2022-08-07T13:49:49.599-0400 [ERROR] vertex “module.databricks_configuration_01.databricks_permissions.token_usage” error: cannot create permissions: tokens tokens does not exist (edited)

Steps to Reproduce

Terraform and provider versions

Terraform v1.2.6
databricks/databricks) v1.2.0

Debug Output

2022-08-07T13:49:49.597-0400 [DEBUG] provider.terraform-provider-databricks_v1.2.0.exe: 404 Not Found {
  “error_code”: “RESOURCE_DOES_NOT_EXIST”,
  “message”: “tokens tokens does not exist”
}: timestamp=2022-08-07T13:49:49.597-0400
2022-08-07T13:49:49.598-0400 [ERROR] provider.terraform-provider-databricks_v1.2.0.exe: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR diagnostic_summary=“cannot create permissions: tokens tokens does not exist” tf_provider_addr=provider tf_req_id=fac59662-67e7-be9f-520a-b7c425ba950e @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/[github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go](http://github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go):56 @module=sdk.proto tf_proto_version=5.3 tf_resource_type=databricks_permissions tf_rpc=ApplyResourceChange timestamp=2022-08-07T13:49:49.598-0400
2022-08-07T13:49:49.599-0400 [ERROR] vertex “module.databricks_configuration_01.databricks_permissions.token_usage” error: cannot create permissions: tokens tokens does not exist (edited)

Important Factoids

Account is in Azure Government Cloud

From the debug log, we can see that the terraform provider was calling this API endpoint for token permissions: provider.terraform-provider-databricks_v1.2.0.exe: PUT /api/2.0/permissions/authorization/tokens

Per our documentation, there is a “preview” in the API endpoint path that was missing in the Terraform provider endpoint. There is a mismatch between the endpoint paths used by Terraform provider and Azure Databricks.

curl -n -X GET "https://<databricks-instance>/api/2.0/preview/permissions/authorization/tokens"

[alexott]

We're using REST API specification, and it doesn't have preview anymore (this change was made more than year ago and worked well). Please clarify with development team why Azure GovCloud still uses old APIs

[nfx]

@qingsun-db it's not clear what resource does not exist. please provide more logs.

please also use azure_environment provider attribute:

provider "databricks" {
    //...
    azure_environment = "usgovernment"
}

https://registry.terraform.io/providers/databricks/databricks/latest/docs#azure_environment

[nfx]

Thank you for the feature request! Currently, the team operates in a limited capacity, carefully prioritizing, and we cannot provide a timeline to implement this feature. Please make a Pull Request if you'd like to see this feature sooner, and we'll guide you through the journey.