modin
modin copied to clipboard
Further optimization for `read_parquet_glob` and `read_pickle_distributed` that create 1 partition from 1 file
As an example, one can use the more complex implementation of read_csv_glob function, which can create several partitions from 1 file if necessary.
However, we need to keep in mind the complexity of support, especially since these are additional public functions that are not available in pandas.