jbrockmendel
jbrockmendel
If pinning an old_freq attr isn't viable, can you make the timezones unnecessary? Could tz_convert to UTC, then do everything with Periods.
> discuss if it's possible and worth doing Surely possible, but daunting. Going back to the old_freq idea, what if you just patched DatetimeIndex's `__getitem__` and `__iter__` to do: ```...
could we get the dtypes from the parquet file metadata and avoid the need to call compute_dtypes later?
can you give an example? im guessing you're referring to pandas.DatetimeTZDtype?
> Could you share the script you are using to profile? I'm not sure that's allowed. If it helps, @yarshev and @anmyachev are looking at the same script.
> Is it possible to at least get a minimum reproducible example? Well, no. I know concat is getting called bc I put a `breakpoint()` inside `MultiIndex.append`. I use `ray...
> Is there the easy way to speed up concatenating the MultiIndex itself on pandas side? There's a patch that speeds up this particular case, but may slow down other...
`pytest modin/pandas/test/` takes me 2.5 hours locally (the pandas test suite with `--skip-slow --skip-db` takes 23 minutes). Of this, test_join_sort takes 55 minutes, test_groupby takes 37, test_series takes 18, test_indexing...
This doesn't look familiar, but I can confirm that pandas 1.5.0 (coming soon) bumps the fsspec minimum supported version to 2021.5.0
most of the pandas arithmetic tests all live in pandas/tests/arithmetic. could try to port them en masse?