google-cloud-python icon indicating copy to clipboard operation
google-cloud-python copied to clipboard

ENH: make use_bqstorage_api True by default

Open tswast opened this issue 6 years ago • 5 comments

It’ll give people the best experience if the fast method is also the default method. Since this requires an additional API to be enabled, it’s technically a breaking change. Similar to how we handled the switch to standard SQL in googleapis/python-bigquery-pandas#195, I propose we complete this work in two phases:

  • [ ] Add use_bqstorage_api to the global context object with a value of False. It is overridable by setting pandas_gbq.context.use_bqstorage_api = True. If the default value of use_bqstorage_api=None is passed into read_gbq, it uses the default value from the global context.
  • [ ] Make a release with this new context value. (Make sure pandas.read_gbq also sends down None for use_bqstorage_api if not supplied.)
  • [ ] Update default value in the global context to True. (Possibly after the BigQuery Storage API goes GA, though maybe sooner.)

tswast avatar Oct 27 '19 04:10 tswast

Any news on when this will be implemented?

ketozhang avatar Mar 12 '21 23:03 ketozhang

Nothing blocking it, now that the API is GA. We probably want to use the create_bqstorage_client parameter in the to_dataframe method for future compatibility.

tswast avatar Mar 15 '21 19:03 tswast

This is long overdue. Bumping the priority on this.

tswast avatar Dec 06 '21 22:12 tswast

Doesn't use_bqstorage_api=True incur more cost? Or is kind of neglible when you're talking roughly 1-10GBs of data?

Et9797 avatar May 15 '24 09:05 Et9797

It's free to read from anonymous tables created by a query.

tswast avatar May 15 '24 21:05 tswast