zarr-python
zarr-python copied to clipboard
zarr-developers
FutureWarning from zarr.open
The following code snippet will raise a FutureWarning:
import numpy
import zarr
group = zarr.open()
group.create('foo', dtype=[('a', int), ('b', int)], fill_value=numpy.zeros((), dtype=[('a', int), ('b', int)])[()], shape=(10, 10))
Problem description
The current behavior raises a FutureWarning:
zarr/util.py:238: FutureWarning: elementwise == comparison failed and returning scalar instead; this will raise an error or perform elementwise comparison in the future.
elif fill_value == 0:
This happens because of the comparison of the utterable dtype to 0 (in line 258) above. I'm not sure if this error is a false positive from numpy or something that will actually break in a future numpy version.
Version and installation information
Please provide the following:
- Value of
zarr.__version__: '2.3.2' - Value of
numcodecs.__version__: '0.6.4' - Version of Python interpreter: 3.7.6
- Operating system (Linux/Windows/Mac): Mac and Linux
- How Zarr was installed: using conda (also from source)
@abergou does using fill_value=0 give you an equivalent result, but without the warning?
import numpy
import zarr
group = zarr.open()
group.create('foo', dtype=[('a', int), ('b', int)], fill_value=0, shape=(10, 10))
Yes in this case it does. If, however, I want to set the fill value to anything other than 0 I get the error (I just used 0 as the minimal example). Thanks!
Ah, I see. Then how about passing a tuple for the fill_value that specifies the fill value for the structured array. For example, here's a slightly modified version of your original snippet which uses a non-trivial fill_value.
import numpy as np
import zarr
group = zarr.open()
group.create('foo',
dtype=[('a', int), ('b', np.float32)],
fill_value=(-1, np.float32(1.23)),
shape=(10, 10),
)
print(group["foo"][:])
Is this more along the lines of what you're looking for?
That works. Thanks!
It also looks like if the fill_value is sourced from a numpy array then it can just be wrapped in tuple:
import numpy
import zarr
group = zarr.open()
group.create('foo', dtype=[('a', int), ('b', int)], fill_value=tuple(numpy.zeros((), dtype=[('a', int), ('b', int)])[()]), shape=(10, 10))
Not that it's important for my purposes at this point, but I wonder if zarr could be made robust to such user inputs by wrapping the fill_value in a tuple. Something like:
if isinstance(fill_value, numpy.void) and fill_value.dtype.names:
fill_value = tuple(fill_value)
Tried this code snippet above to check if the issue remains
import numpy
import zarr
group = zarr.open()
group.create('foo', dtype=[('a', int), ('b', int)], fill_value=numpy.zeros((), dtype=[('a', int), ('b', int)])[()], shape=(10, 10))
Future Warning still rises.
zarr\util.py:259: FutureWarning: elementwise == comparison failed and returning scalar instead; this will raise an error or perform elementwise comparison in the future.
elif fill_value == 0:
Doh. Sorry, @maiia-pi, I somehow lost a comment I had prepared. More briefly, it looks like @d-v-b ran into this issue here:
https://github.com/zarr-developers/zarr-python/pull/738/files#diff-f389c1f0162cd60b1b5a4187dad20dfd3d22c98da2ebe22e860ea830bc25a0e9R692
You might try to use his zarr.util.all_equal method and see if it will fix it, but I assume that this is a bigger problem if we didn't have a suggestion.
