python-bigquery-pandas icon indicating copy to clipboard operation
python-bigquery-pandas copied to clipboard

Published 20 hours ago • verified publisher icon googleapis

Expose `dtype_backend` argument to align with other I/O methods

Open tswast opened this issue 2 years ago • 0 comments

dtypes_backend options:

  • "default" (current "mixed" behavior, where most extension dtypes are used but not Float64Dtype or StringDtype) -- edit: might want to check dtype_backend in pandas global options (per https://github.com/pandas-dev/pandas/issues/51433),
  • None (all legacy types),
  • "numpy_nullable" corresponds to existing behavior but also nullable floats (https://github.com/googleapis/python-bigquery-pandas/issues/618) and strings (somewhat related: https://github.com/googleapis/python-bigquery-pandas/issues/275#issuecomment-566100550).
  • "pyarrow" (all Arrow dtypes)

Implementation note: we probably need to call to_arrow for all but default, where to_dataframe is still OK.

See also: https://github.com/pandas-dev/pandas/pull/51853

tswast avatar Mar 28 '23 19:03 tswast