Anatoly Myachev
Anatoly Myachev
> .iloc on modin is indeed slower than pandas, but as soon as we're speaking about execution times less than 0.1 second, we can do nothing about it (because of...
Here we are already trying to skip the masking stage: https://github.com/modin-project/modin/blob/097ea527c8e3f099e1f252b067a1d5eb055ad0b5/modin/core/dataframe/pandas/dataframe/dataframe.py#L1181 Apparently some index cache is missing.
Hello @JacobKwon! Thanks for your contribution and sorry for the long response. First of all, I would like to clarify if square brackets work if you are using pandas and...
@JacobKwon I can reproduce the problem and seem to have found the cause. Modin uses `fsspec` library in cases when pandas doesn't: https://github.com/fsspec/filesystem_spec/issues/1476
@YarShev could you show where the implementation is? I see only https://github.com/modin-project/modin/blob/31f8bd0e699b7b56014ac37b67f282687c8d6b43/modin/core/storage_formats/base/query_compiler.py#L1439
@mvashishtha I agree, we should change the default behaviour.
The reason seems to be the same as in #6865 (`to_feather` that defaults to pandas)
This code can be used as a starting point: ```python diff --git a/modin/core/dataframe/pandas/dataframe/dataframe.py b/modin/core/dataframe/pandas/dataframe/dataframe.py index c1e990e2..b17a2596 100644 --- a/modin/core/dataframe/pandas/dataframe/dataframe.py +++ b/modin/core/dataframe/pandas/dataframe/dataframe.py @@ -174,6 +174,7 @@ class PandasDataframe(ClassLogger): # These properties...
A short summary: * `"s3://amazon-reviews-pds/parquet/product_category=Wireless/"` is no longer a public dataset * `os.environ["__MODIN_AUTOIMPORT_PANDAS__"] = "1"` - using this [will slow down](https://github.com/modin-project/modin/issues/5157#issuecomment-1500225150) rather than speed up. [More details](https://github.com/modin-project/modin/blob/92741feda50a34783e4c6adc3cd46a8a30f0cae2/docs/getting_started/troubleshooting.rst#poor-performance-of-the-first-operation-with-modin-on-ray-engine). * > I...