Ryan Abernathey

Big 👍 to that idea.

We now have automatic detection of the ArrayBytes codec based on dtype: https://github.com/zarr-developers/zarr-python/blob/395604dcc86b9154ea673d15d9c9bb9a74db1538/src/zarr/codecs/__init__.py#L37-L46 Next step for this issue is to just add a default BytesBytes compressor.

> in which cases should we use VLenBytes and in which cases VLenUTF8? VLenUTF8 is explicitly for strings. VLenBytes is just any random bytes. They are mutually exclusive.

Maybe @martindurant has some insights?

I actually started something like this three years ago! https://github.com/rabernat/mds2zarr - of course VirtualiZarr is much better and more robust approach. I agree with @cspencerjones that the funky compression of...

Except it's really complicated because the "codec" for decoding each array relies on an external dataset (the null mask) which doesn't even have the same shape as the data. This...

Thanks for the ping @mwlon! Yes we should make a release. I have been doing some benchmarking of pcodec from the master branch in the meantime.

@jakirkham - ~what is the release process for numcodecs? I'd be happy to make a release so we can get pco out.~ Edit: I figured it out: https://numcodecs.readthedocs.io/en/stable/contributing.html#release-procedure Are you...

v0.13.0 is on pypi: https://pypi.org/project/numcodecs/0.13.0/ 😅 I followed the [release procedure](https://numcodecs.readthedocs.io/en/stable/contributing.html#release-procedure) and it seemed to work fine. The one thing I did not do was create an actual release on...

Correct, but I personally don't know how to make it update. I thought RTD would automatically detect a new tag, but that hasn't happened. We probably need help from @jakirkham.