Phillip Cloud
Phillip Cloud
Looks like the docs preview are showing a zoom that doesn't match the current.
Done in #9350
@binste Thanks for digging into this. Interesting results! Can you share the query that's now taking 11 seconds with Ibis 9.0?
Regarding drop and relocate, they're both implemented using a somewhat naive approach: - `drop`: project every column except the input columns - `relocate`: basically a reprojection that loops through every...
@binste Any chance you can try with a pre-release wheel? Benchmarking locally (I'll PR this code in a bit) I do still see linear scaling, but the constant factor isn't...
@binste The pre-release wheels are published every Sunday, so give it a go as soon as you're able to using something like this to install a pre-release wheel with `ibis-framework`...
@binste Following up here, would you be able to profile your code again and determine whether relocate and rename are still performing undesirably?
@ncclementi Thanks! Yeah, I see slightly different results for `main`: ``` | 10 | 0.0028498172760009766 seconds | 20 | 0.0045757293701171875 seconds | 50 | 0.011339426040649414 seconds | 100 | 0.02110433578491211...
Using `pyinstrument`, I see about a 2x speedup in `rename` from 8.0.0 to 9.1.0: script: ```python from __future__ import annotations import ibis num_cols = 1000 t = ibis.table(name="t", schema=[(f"MyCol{i}", "int")...
I think there are a couple things left to explore here. `rename` and `relocate` in the worst case need to look at every column, and that's currently the bottleneck AFAICT,...