Tom White
Tom White
What's the error that you get? I'm running with Python 3.11 on an M1 Mac and haven't seen this.
Closing, since `concatenate2` was removed in #605
Thanks for doing this work @TomAugspurger! Coincidentally, I've been looking at memory overheads for Zarr storage operations across different filesystems (local/cloud), compression settings, and Zarr versions: https://github.com/tomwhite/memray-array > There are...
I wonder if any of the [memory management](https://arrow.apache.org/docs/python/memory.html) machinery that has been developed for Apache Arrow would be of use here?
> 2\. For the compressed case, we might be able to improve things once we know the size of the output buffers. I see that numcodec's `zstd.decode` takes an output...
> Zarr Python v2 actually does this already: https://github.com/zarr-developers/zarr-python/blob/support/v2/zarr/core.py#L2044-L2050 FYI here's my hacky attempt to do something similar in v3: https://github.com/tomwhite/zarr-python/commit/9b5e7fc7d1a7e283917f04f04ebe3f60a46055c2
It's worth doing #715 and/or #731 first as they are simpler to do and may well give similar performance and memory improvements as using key-value store (e.g. obstore) directly from...
Also `cumulative_prod`
Also affecting https://github.com/cubed-dev/cubed/actions/workflows/zarr-v2-tests.yml, but only intermittantly.
See #762