DataflowTemplates icon indicating copy to clipboard operation
DataflowTemplates copied to clipboard

[Feature Request]: Allow specifying withQueryTempDataset for the BQ query

Open dmedora opened this issue 1 year ago • 0 comments

Related Template(s)

bigquery-to-bigtable

What feature(s) are you requesting?

BigQueryIO fromQuery tries to create a temporary dataset to materialize query results into. This means the executing service accounts must have the bigquery.datasets.create permission. Alternatively, there is a withQueryTempDataset method that allows specifying a dataset to use so that the creation permission is not required (BigQueryIO docs).

It would be great if this template / BigQueryReadOptions supported a queryTempDataset parameter to avoid granting excess permissions to the service account.

dmedora avatar Sep 30 '24 22:09 dmedora