Support BigQuery oauth credentials

[lewish]

Currently BigQuery can only be authorized through a service account, which adds significant overhead to setting up a BQ project.

It should be possible to provide oauth credentials when executing against BigQuery.

• During init-creds add an option to use OAuth for BigQuery projects

• Send the user to the oauth connection page with a callback to localhost:someport

• Spin up a small server to handle the oauth callback

• Write the oauth credentials to the .df-credentials.json file

• [ ] Create a Dataform OAuth client ID

• [ ] Support OAuth credentials in the BigQuery adapter and extend the credentials proto to store this

• [ ] Implement the new init-creds flow above

[dorzey]

👍 we would definitely like to have this

[thinhha]

Downloading service account keys is not very good security practice. The better approach is to use OAUTH as described in this article: https://medium.com/@jryancanty/stop-downloading-google-cloud-service-account-keys-1811d44a97d9

One way to do this is to use GOOGLE_APPLICATION_CREDENTIALS as described in: https://github.com/dataform-co/dataform/issues/1164

Another way is for Dataform to expect a short-lived OAuth 2.0 access token, either as input to the CLI or from an environment variable like as done by terraform: https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#access_token

[jamiekt]

I wholeheartedly agree with @thinhha , access tokens would be an ideal mechanism here.

[jamiekt]

This is a very relevant reference: https://cloud.google.com/iam/docs/creating-short-lived-service-account-credentials

[thinhha]

more details on how DBT did this here: https://github.com/fishtown-analytics/dbt/issues/2672