dbt-databricks
dbt-databricks copied to clipboard
Databricks OAuth secret not working as expected in 1.8
trafficstars
Describe the bug
We are sometimes running dbt as a service principal from the command line. profiles.yml then looks something like
project:
outputs:
o1:
auth_type: oauth
client_id: [sp_id]
client_secret: "{{ env_var('DB_CLIENT_SECRET') }}"
type: databricks
[...]
The value of DB_CLIENT_SECRET has been a Databricks OAuth secret as per the docs. This has been working fine with dbt-databricks>=1.7 but now when upgrading to 1.8(.5) it starts failing with
Runtime Error
invalid_client: AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '[sp_id]'.
And if I instead supply the client secret that I set up in Microsoft Entra ID it works again.
What is the intended usage? Do I need to switch from using Databricks OAuth secrets here to using that from Microsoft Entra ID?