dbt-databricks icon indicating copy to clipboard operation
dbt-databricks copied to clipboard

Not possible to OAuth as authentication type in setup with SQL Serverless

Open excavator-matt opened this issue 6 months ago • 2 comments

Describe the bug

I tried to follow the official setup guide and connect it to my Serverless SQL Warehouse. The documentation clearly states that OAuth is preferred, but the installation script (dbt init) only allows for access token.

To choose an authentication type, enter the number that corresponds with use oauth (recommended) or use access token.

[1] use access token
Desired access token option (enter a number): 

This could mean one of the following things

  1. The documentation didn't mention that OAuth isn't supported for serverless SQL warehouses
  2. The documentation/Python package didn't mention how to enable OAuth
  3. The installation script needs to fixed so that it allows for OAuth as recommended by the tutorial.

Steps To Reproduce

Install latest version (dbt-databricks==1.10.4), try connecting it to a SQL Serverless Warehouse (AWS).

Expected behavior

It should be possible to choose OAuth as the documentation suggests.

Screenshots and log output

[1] use access token
Desired access token option (enter a number): 

System information

The output of dbt --version:

dbt --version
WARNING:thrift.transport.sslcompat:using legacy validation callback
.venv/lib/python3.13/site-packages/pydantic/_internal/_config.py:373: UserWarning: Valid config keys have changed in V2:
* 'allow_population_by_field_name' has been renamed to 'validate_by_name'
  warnings.warn(message, UserWarning)
Core:
  - installed: 1.10.2
  - latest:    1.10.2 - Up to date!

Plugins:
  - databricks: 1.10.4 - Up to date!
  - spark:      1.9.2  - Up to date!

The operating system you're using: macOS Sequoia 15.5 (24F74)

The output of python --version: Python 3.13.5

excavator-matt avatar Jun 30 '25 08:06 excavator-matt

Installation script needs to be fixed. Thanks for reporting.

benc-db avatar Jun 30 '25 17:06 benc-db

Please reference here in the mean time: https://docs.getdbt.com/docs/core/connect-data-platform/databricks-setup#authentication-parameters The doc is a little confusing but basically if you set oauth as the type, and set no other auth fields, you'll get U2M Oauth using the Databricks OAuth app. If you want to do M2M with Databricks OAuth, specify the client_id and client_secret, and if you want to do M2M with Azure OAuth (i.e. with an Azure app registration), specify the azure_client_id and azure_client_secret.

benc-db avatar Jul 02 '25 18:07 benc-db