databricks-sdk-go icon indicating copy to clipboard operation
databricks-sdk-go copied to clipboard

[ISSUE] Cloud KMS region `europe` or `global` not support for Customer Managed keys

Open ChristianGroentved opened this issue 11 months ago • 0 comments

Description When trying to use databricks_mws_customer_managed_keys with a keyring that has either a global or europe as location I get an error that the region isn't supported

Reproduction

provider "databricks" {
  alias                  = "accounts"
  host                   = "https://accounts.gcp.databricks.com"
  google_service_account ="<EMAIL>"
  account_id             = "<ACCOUNT_ID>"
}
terraform {
  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = ">=1.51.0"
    }
    google = {
      source  = "hashicorp/google"
      version = ">=6.0.1"
    }
  }
}
data "google_kms_key_ring" "dbx_key_ring" {
  project  = "<PROJECT>"
  name     = "databricks"
  location = "europe"
}
data "google_kms_crypto_key" "storage_crypto_key" {
  name     = "dev"
  key_ring = data.google_kms_key_ring.dbx_key_ring.id
}
resource "databricks_mws_customer_managed_keys" "dbx_workspace_storage" {
  account_id = "<ACCOUNT_ID>"
  gcp_key_info {
    kms_key_id = data.google_kms_crypto_key.storage_crypto_key.id
  }
  use_cases = ["STORAGE"]
  lifecycle {
    ignore_changes = all
  }
}

Expected behavior Able to utilise keyrings that cover more than one region

Debug Logs

Error: cannot create mws customer managed keys: unexpected error handling request: invalid character 'B' looking for beginning of value. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:
│ ```
│ POST /api/2.0/accounts/redacted/customer-managed-keys
│ > * Host: 
│ > * Accept: application/json
│ > * Authorization: REDACTED
│ > * Content-Type: application/json
│ > * Traceparent: 00-357513985a530a86698b98dbbaf9107b-767f1dff94713124-01
│ > * User-Agent: databricks-tf-provider/1.51.0 databricks-sdk-go/0.45.0 go/1.22.6 os/darwin terraform/1.9.8 resource/mws_customer_managed_keys auth/google-id
│ > * X-Databricks-Gcp-Sa-Access-Token: redacted
│ > {
│ >   "account_id": "redacted",
│ >   "gcp_key_info": {
│ >     "kms_key_id": "projects/redacted/locations/global/keyRings/test_dbx/cryptoKeys/redacted"
│ >   },
│ >   "use_cases": [
│ >     "STORAGE"
│ >   ]
│ > }
│ < HTTP/2.0 400 Bad Request
│ < * Content-Type: text/plain; charset=utf-8
│ < * Date: Wed, 6 Nov 2024 08:38:24 GMT
│ < * Server: databricks
│ < * Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
│ < * Vary: Accept-Encoding
│ < * X-Content-Type-Options: nosniff
│ < * X-Request-Id: d4b8e3c2-f81e-4da4-bdbc-134d8f8a047c
│ < BAD_REQUEST: GCP region global is invalid. Please check documentation for supported regions.
│ ```
│ 
│   with module.trail_create.databricks_mws_customer_managed_keys.dbx_workspace_storage,
│   on .terraform/modules/trail_create/modules/deploy_workspace/main.tf line 63, in resource "databricks_mws_customer_managed_keys" "dbx_workspace_storage":
│   63: resource "databricks_mws_customer_managed_keys" "dbx_workspace_storage" {
│ 
╵

ChristianGroentved avatar Nov 06 '24 08:11 ChristianGroentved