modin icon indicating copy to clipboard operation
modin copied to clipboard

``dataframe.dropna()`` doesn`t work correctly at MODIN_STORAGE_FORMAT=omnisci

Open Rubtsowa opened this issue 3 years ago • 2 comments

When I run:

import pandas
import modin.pandas as pd
import modin.config as cfg
cfg.StorageFormat.put('omnisci')
from modin.pandas.test.utils import (test_data_values, axis_values)

data = test_data_values[1]
axis = axis_values[2]
modin_df = pd.DataFrame(data)
pandas_df = pandas.DataFrame(data)
modin_result = modin_df.dropna(axis=axis)
pandas_result = pandas_df.dropna(axis=axis)
df_equals(modin_result, pandas_result)

I have error:

AssertionError: DataFrame are different

           DataFrame shape mismatch
           [left]:  (0, 0)
           [right]: (256, 0)

Rubtsowa avatar Jan 11 '22 18:01 Rubtsowa

@modin-project/modin-omnisci could someone please check if this still happens?

vnlitvinov avatar Aug 29 '22 13:08 vnlitvinov

Should be fixed by #4910

AndreyPavlenko avatar Sep 21 '22 11:09 AndreyPavlenko

@AndreyPavlenko, #4910 is merged. Should we close this issue?

YarShev avatar Sep 27 '22 18:09 YarShev

The issue is not reproducible after the merge, just the following warning is printed: UserWarning: HDK's dropna does not support 'thresh' and 'axis' parameters. is not currently supported by ExperimentalHdkOnNative, defaulting to pandas implementation.

AndreyPavlenko avatar Sep 27 '22 18:09 AndreyPavlenko

Okay, that looks like a separate feature to implement so I am closing this issue.

YarShev avatar Sep 27 '22 18:09 YarShev