modin icon indicating copy to clipboard operation
modin copied to clipboard

Further optimization for `read_parquet_glob` and `read_pickle_distributed` that create 1 partition from 1 file

Open anmyachev opened this issue 1 year ago • 0 comments

As an example, one can use the more complex implementation of read_csv_glob function, which can create several partitions from 1 file if necessary.

However, we need to keep in mind the complexity of support, especially since these are additional public functions that are not available in pandas.

anmyachev avatar Jan 12 '24 22:01 anmyachev