narwhals
narwhals copied to clipboard
test: use `dask` with pyarrow dtype backend for testing
There's odd behaviour in dask when using the classical types, which is quite a pain to work around in tests. I think we should just test using their pyarrow backend reduce the amount of gymnastics we have to do in tests
Tried the simplest change:
def dask_lazy_p2_constructor(obj: Data) -> NativeDask: # pragma: no cover
import dask.dataframe as dd
+ import pandas as pd
- return cast("NativeDask", dd.from_dict(obj, npartitions=2))
+ pd_frame = pd.DataFrame(obj).convert_dtypes(dtype_backend="pyarrow")
+ return cast("NativeDask", dd.from_pandas(pd_frame, npartitions=2))
There is one xpass, and 44 failures 😭