exdir
exdir copied to clipboard
Should object arrays be allowed
We should decide if we should allow object arrays or not. There are several advantages and disadvantages to this. Currently we allow object arrays, but we have not made a conscious decision to support them. They just happen to work.
They should not be allowed and no longer seem to work because memmap fails in this case. I think we should catch this before memmap does and just check dtype.hasobject. If true, we raise an exception saying that this is not supported.
Also not supported w. h5py:
In [8]: f = h5py.File('objecttest.h5', 'w')
In [9]: f['o'] = np.array([object()])
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-9-ff5a86282ebb> in <module>()
----> 1 f['o'] = np.array([object()])
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/_hl/group.pyc in __setitem__(self, name, obj)
289
290 else:
--> 291 ds = self.create_dataset(None, data=obj, dtype=base.guess_dtype(obj))
292 h5o.link(ds.id, self.id, name, lcpl=lcpl)
293
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/_hl/group.pyc in create_dataset(self, name, shape, dtype, data, **kwds)
106 """
107 with phil:
--> 108 dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
109 dset = dataset.Dataset(dsid)
110 if name is not None:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/_hl/dataset.pyc in make_new_dset(parent, shape, dtype, data, chunks, compression, shuffle, fletcher32, maxshape, compression_opts, fillvalue, scaleoffset, track_times)
95 else:
96 dtype = numpy.dtype(dtype)
---> 97 tid = h5t.py_create(dtype, logical=1)
98
99 # Legacy
h5py/h5t.pyx in h5py.h5t.py_create()
h5py/h5t.pyx in h5py.h5t.py_create()
h5py/h5t.pyx in h5py.h5t.py_create()
TypeError: Object dtype dtype('O') has no native HDF5 equivalent
Good point. I think we should also raise a TypeError
to have as close to the same behavior as h5py as possible. That eases porting from h5py to Exdir.