python-bigquery-dataframes icon indicating copy to clipboard operation
python-bigquery-dataframes copied to clipboard

Possible to use read_gbq() without bigquery.readsessions.create permissions?

Open chrisperks opened this issue 1 year ago • 2 comments

Hi!

I'm using bigframes 0.19.2 and in particular, using pandas.read_gbq(..) to read an SQL statement to a Pandas DataFrame.

I don't have permissions to create a ReadSession on our BigQuery instance, so my query fails, reporting I don't have the relevant bigquery.readsessions.create permissions.

When using the regular BigQuery client, I can set the option create_bqstorage_client=False to get around this lack of permissions.

Can I use read_gbq in any similar way, or otherwise set options to remove the need for bigquery.readsessions.create permissions?

Thanks. Chris.

chrisperks avatar Jan 23 '24 13:01 chrisperks

Thanks for the feedback @chrisperks. At the moment it is not possible to avoid using the BigQuery Storage Read API in BigQuery DataFrames to_pandas and other methods that could download data in bulk like __repr__ or .values.

Performance won't be ideal without the BQ Storage API, but it should be possible, as you have noted the BQ client library does support this.

I propose we add global use_bqstorage_read_api=True|False option to https://cloud.google.com/python/docs/reference/bigframes/latest/bigframes._config.bigquery_options.BigQueryOptions to avoid the BQ Storage API in I/O.

tswast avatar Jan 23 '24 17:01 tswast

Thanks for the feedback!

chrisperks avatar Jan 26 '24 09:01 chrisperks