dbt-databricks icon indicating copy to clipboard operation
dbt-databricks copied to clipboard

Databricks OAuth secret not working as expected in 1.8

Open markus-sh-lftt opened this issue 1 year ago • 4 comments
trafficstars

Describe the bug

We are sometimes running dbt as a service principal from the command line. profiles.yml then looks something like

project:
  outputs:
    o1:
      auth_type: oauth
      client_id: [sp_id]
      client_secret: "{{ env_var('DB_CLIENT_SECRET') }}"
      type: databricks
      [...]

The value of DB_CLIENT_SECRET has been a Databricks OAuth secret as per the docs. This has been working fine with dbt-databricks>=1.7 but now when upgrading to 1.8(.5) it starts failing with

Runtime Error
  invalid_client: AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '[sp_id]'.

And if I instead supply the client secret that I set up in Microsoft Entra ID it works again.

What is the intended usage? Do I need to switch from using Databricks OAuth secrets here to using that from Microsoft Entra ID?

markus-sh-lftt avatar Aug 08 '24 09:08 markus-sh-lftt