pyo3-polars icon indicating copy to clipboard operation
pyo3-polars copied to clipboard

Forced rechunking

Open aldanor opened this issue 1 year ago • 8 comments

This took me a while to figure out (since this was the last place I'd expect a forced rechunk to happen) - while passing huge frames from Python to Rust and back, noticed that they end up arriving in one chunk even if they were multi-chunked originally.

Is there any reason to not leave rechunking to the end-user? (since in some cases it may end up being very detrimental)

https://github.com/pola-rs/pyo3-polars/blob/0165cb4f6795bf9a08fe04473aba408a02119b91/pyo3-polars/src/lib.rs#L121

... and also this: https://github.com/pola-rs/pyo3-polars/blob/0165cb4f6795bf9a08fe04473aba408a02119b91/pyo3-polars/src/lib.rs#L163

aldanor avatar Oct 06 '23 00:10 aldanor