python-bigquery-pandas icon indicating copy to clipboard operation
python-bigquery-pandas copied to clipboard

Support for coercing to floats

Open rjrudin opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe.

This is related to #474 , as we try to migrate from pandas.read_sql_query to either pandas.read_gbq or pandas_gbq.read_gbq. The issue here is that we use read_sql_query with coerce_float=True, but with read_gbq, our tests that expect float conversion.

Describe the solution you'd like

We'd like the equivalent of coerce_float as an argument for read_gbq.

Describe alternatives you've considered

We are looking into whether we can not assume the use of coerce_float=True, but in the short term, having coerce_float support would allow us to preserve our existing functionality and gain the significantly increased performance of read_gbq.

We also looked into using the dtypes argument in pandas_gbq.read_gbq, but that's not viable for us in scenarios where we don't know the schema ahead of time - i.e. a user provides connection details and says "Hey, read a dataframe from here". It would be feasible to check out the table schema and infer a dtypes argument based on it, but life would of course be a lot simpler if we could do what we're doing with pandas.read_sql_query and rely on coerce_float.

Additional context

N/A

rjrudin avatar Jan 24 '22 19:01 rjrudin