data-infra
data-infra copied to clipboard
Investigate: Failed dbt OAuth to BigQuery for Some JupyterHub Users
Describe the bug
Before moving to a local development setup, a recent new user in the Cal-ITP ecosystem (a contractor, not a Caltrans staffer) attempted to get set up to develop with dbt in JupyterHub. She experienced timeouts while trying to run dbt debug
and other basic dbt commands, with errors indicating failed authentication, even though she was able to connect to BigQuery and run queries by setting up a connection in a Jupyter notebook.
An initial investigation failed to turn up an obvious user misconfiguration on the GCP side, and other users are able to use dbt commands without trouble inside the JupyterHub environment. We believe that either A.) the pathways dbt uses to fetch and send auth information when run within JupyterHub are somehow failing for this user, or B.) this user's OAuth setup in GCP is limited in some way vs. other users that prevents a successful connection via the JupyterHub dbt workflow.
To Reproduce We do not know yet whether this issue will impact other future JupyterHub users, so at the time of issue creation this Slack thread provides the most detailed description of the behavior. It specifically arises when using dbt commands from JupyterHub, and doesn't arise when using other methods to interact with BigQuery.
Expected behavior All users should be able to use dbt within JupyterHub to work with the data warehouse.