modin icon indicating copy to clipboard operation
modin copied to clipboard

BUG: `Dataframe.astype(...)` Fails With `'bool' object has no attribute 'all'`

Open zombie-einstein opened this issue 6 months ago • 2 comments

Modin version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest released version of Modin.

  • [ ] I have confirmed this bug exists on the main branch of Modin. (In order to do this you can follow this guide.)

Reproducible Example

import modin.pandas as pd
import numpy as np

# This will cast to a float type due to NaNs
df = pd.DataFrame({"a": [1, 2, 3, np.nan]})

# This passes
df.astype({"a": pd.Int64Dtype()})
>>    a
0     1
1     2
2     3
3  <NA>

# This fails
df.astype(pd.Int64Dtype())
>> AttributeError: 'bool' object has no attribute 'all'

Issue Description

I think this is the same error as #7276, but that issue was closed. When using astype on a Dataframe it fails at this check

File .../modin/core/dataframe/pandas/dataframe/dataframe.py:
   1736, in PandasDataframe.astype(self, col_dtypes, errors)
   1730         return df.astype(
   1731             {k: v for k, v in col_dtypes.items() if k in df}, errors=errors
   1732         )
   1734 else:
   1735     # Assume that the dtype is a scalar.
-> 1736     if not (col_dtypes == self_dtypes).all():
   1737         new_dtypes = self_dtypes.copy()
   1738         new_dtype = pandas.api.types.pandas_dtype(col_dtypes)

AttributeError: 'bool' object has no attribute 'all'

When the type argument is a single value (i.e. astype(pd.Int64Dtype())) then it seems that col_dtypes == self_dtypes works out as a single bool value (hence no all attribute).

Note that this works Ok if the argument is a dictionary of column namess to dtypes.

This also seems to be the same for Series, i.e.:

df["a"].astype(pd.Int64Dtype())

Fails with the same error

Expected Behavior

In native Pandas

df.astype(pd.Int64Dtype())

casts the DataFrame/series to the argument type

Error Logs

No response

Installed Versions

'0.31.0'

zombie-einstein avatar Aug 07 '24 18:08 zombie-einstein