Support Authenticating using a Service Principal with Open ID Connect

[KarthicPortal]

Hi there,

Currently the supported authentication types are pat, basic, azure-client-secret, azure-msi, azure-cli, google-credentials, and google-id for databricks provider.

Proposal With the introduction of Workload Identity federation in CI/CD pipelines for Azure Resource Manager, need azure-oidc support as well to authenticate azure databricks provider. This support is already available for AzureRM provider. When we combine both azurerm and azure-databricks providers for deployment in single CI/CD pipeline, it is hard to maintain different authentication types for each provider.

References https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_oidc

[tmh-msg]

any updates when this will be added?

[juicybaba]

Hello? 😃

[davidzenisu]

We're having high demand for this as well! With workload identity federation now being the default setup for service connection this requires us to maintain legacy configurations.

[ganesh-sahastrabuddhe]

@alexott any update on this issue?

[Chambras]

Please! can we get an update on this?

[KarthicPortal]

Hi All,

Meanwhile this is how I am working around for the both azurerm and azure-databricks providers to authenticate in single terraform state file (maybe it helps someone):

For azurerm,

   ARM_USE_OIDC=true
   ARM_CLIENT_ID=xxxx
   ARM_SUBSCRIPTION_ID=xxx
   ARM_TENANT_ID=xxx

For azure-databricks,

   DATABRICKS_AUTH_TYPE="pat"
   DATABRICKS_TOKEN=$(az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d --query "accessToken" -o tsv)
   DATABRICKS_HOST=xxx

[MihalisW]

+1 on this request

[harsha1163]

+1 on this request

[harsha1163]

Hi All,

Meanwhile this is how I am working around for the both azurerm and azure-databricks providers to authenticate in single terraform state file (maybe it helps someone):

For azurerm,

   ARM_USE_OIDC=true
   ARM_CLIENT_ID=xxxx
   ARM_SUBSCRIPTION_ID=xxx
   ARM_TENANT_ID=xxx

For azure-databricks,

   DATABRICKS_AUTH_TYPE="pat"
   DATABRICKS_TOKEN=$(az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d --query "accessToken" -o tsv)
   DATABRICKS_HOST=xxx

Instead of using databricks access token, as you are already referencing the arm client secrets as environment variables. In the terraform databricks provider block you can use the ‘auth_type=“azure_client_secret’ argument.

[alexott]

For Azure, just use azure-cli auth type and omit DATABRICKS_TOKEN - then TF provider will automatically call az account and generate/refresh token

[audunsolemdal]

For Azure, just use azure-cli auth type and omit DATABRICKS_TOKEN - then TF provider will automatically call az account and generate/refresh token

It's been a while since I tried, but unless something has changed there are still some issues with this approach. Ideally what we would want is an explicit flag which forces the auth mode to be OIDC, e.g. ARM_USE_OIDC

https://github.com/databricks/terraform-provider-databricks/issues/2234 https://github.com/databricks/databricks-sdk-go/issues/495

Long story short, when these env vars are set ARM_CLIENT_ID, ARM_TENANT_ID (they are in use by my azurerm and azuread providers) it is not possible to use the built-in OIDC-auth for the databricks provider as there are confliciting arguments.

[mateuszte]

Any update here?

[audunsolemdal]

OIDC works fine now without any need for workarounds on newer versions of the databricks terraform provider. Unsure when it started working exactly, but it was after 1.49.x

provider "databricks" {
  host = "https://xxxxxx.azuredatabricks.net"
}

provider "databricks" {
  host       = "https://accounts.azuredatabricks.net"
  account_id = "xxxx"
  alias      = "account"
}

You can either set the env vars ARM_CLIENT_ID, ARM_TENANT_ID

or you can set the following properties in the provider block

azure_client_id, azure_tenant_id

https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/azure-authenticate-with-oidc