polars
polars copied to clipboard
0.15.16 made pyarrow a requirement when using pl.from_pandas
Polars version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of Polars.
Issue description
It seems like pyarrow is required now when using pl.from_pandas
. 2 days ago this was not true. I believe it is due to this change, but I am not 100% sure:
- default to pyarrow for writing parquet (#6313)
I could add pyarrow to my dependencies but I am not sure if you intended to make it a requirement as I see a lot of python code to check if pyarrow exists.
Reproducible example
python
import polars as pl
import pandas as pd
polars_df = pl.from_pandas(pd.DataFrame([{"foo": "bar"}]))
print(polars_df)
Expected behavior
Use default writer and not pyarrow.
Installed versions
0.15.16
For convertion to and from pandas pyarrow was already a requirement.
This is the code we run:
"""Construct a PyDataFrame from a pandas DataFrame."""
length = data.shape[0]
arrow_dict = {
str(col): _pandas_series_to_arrow(
data[col], nan_to_none=nan_to_none, min_len=length
)
for col in data.columns
}
arrow_table = pa.table(arrow_dict)
return arrow_to_pydf(
arrow_table, schema=schema, schema_overrides=schema_overrides, rechunk=rechunk
)
I will close this as this has always been a requirement.
Makes sense. Thanks for taking a look at this @ritchie46 !