TypeError: Cannot interpret 'Float32(endianness='little')' as a data type

[rsignell]

My virtualizarr append-to-icechunk notebook was working with virtualizarr=2.1.1 but is broken with virtualizarr=2.1.2: https://gist.github.com/rsignell/553ac0f3025413288d314897c8fa4ca1

In cell [23] it errors out with TypeError: Cannot interpret 'Float32(endianness='little')' as a data type

Is this something I need to change or is it a bug?

This specific append workflow isn't reproducible without credentials, but I'd be happy to supply them. The notebook that creates the repo works fine with 2.1.2, in case that provides any additional clues: https://gist.github.com/rsignell/b6acafa34c7872b35d0fe9ffcd0fa7b3

[rsignell]

@keewis, might this be related to your endianness change in 2.1.2?

[rsignell]

Okay, @keewis helped me figure out that what broke my workflow was https://github.com/zarr-developers/VirtualiZarr/pull/760 (so I was wrong -- it had nothing to do with his change!)

When I add back the .to_native_dtype() it works as expected.

[TomNicholas]

Thanks for the report and investigation @rsignell ! I'll revert that PR now, and notify @d-v-b and @maxrjones because I don't really understand what happened there.

[maxrjones]

@rsignell which version of Zarr are you using?

[keewis]

From what I remember, it's the most recent one: 3.1.3

[rsignell]

(notebook) jovyan@jupyter-rsignell4:~$ conda list zarr
# packages in environment at /srv/conda/envs/notebook:
#
# Name                    Version                   Build  Channel
virtualizarr              2.1.2              pyhd8ed1ab_0    conda-forge
zarr                      3.1.3              pyhcf101f3_0    conda-forge