pynwb
pynwb copied to clipboard
NeurodataWithoutBorders
[Bug]: Cannot add electrodes to existing nwbfile in Zarr format
What happened?
We have a pipeline that adds LFP data after an NWB file with an electrodes table is created. The LFP electrodes should be appended to the electrodes table, but this is causing errors because hdmf cannot handle the zarr.Array object (see error in traceback)
Seems to be due to the fact that the hdmf.data_utils.append_data function is called by pynwb, which doesn't handle Zarr arrays.
@oruebel @rly @mavaylon1 could you take a look? Should the hdmf.data_utils.append_data handle the zarr.Array case?
Steps to Reproduce
# create mock NWB file
from pynwb.testing.mock.file import mock_NWBFile
from pynwb.file import Device
from pynwb.ecephys import ElectrodeGroup
nwbfile = mock_NWBFile()
# add device
device = Device(name="my_device", description="my device")
_ = nwbfile.add_device(device)
# add electrode group
eg = ElectrodeGroup(name="tetrode", description="my_tetrode", location="unknown", device=device)
_ = nwbfile.add_electrode_group(eg)
nwbfile.add_electrode_column(name="source", description="1st or 2nd round")
# add a few electrodes
rel_xs = [0., 10., 20., 30.]
rel_ys = [0., 0., 0., 0.]
for x, y in zip(rel_xs, rel_ys):
for x, y in zip(rel_xs, rel_ys):
nwbfile.add_electrode(
rel_x=x,
rel_y=y,
enforce_unique_id=True,
source="first",
group=eg,
location="unknown"
)
# save to Zarr (same error in "a" mode)
with NWBZarrIO("test.nwb", mode="w") as io:
io.write(nwbfile)
# now reload and try to add some more electrodes
io = NWBZarrIO(str(results_folder / "test.nwb"), mode="r")
nwbfile_read = io.read()
rel_xs = [50., 60., 70., 80.]
rel_ys = [0., 0., 0., 0.]
for x, y in zip(rel_xs, rel_ys):
nwbfile_read.add_electrode(
rel_x=x,
rel_y=y,
enforce_unique_id=True,
source="second",
group=eg,
location="unknown"
)
Traceback
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[69], line 5
2 rel_ys = [0., 0., 0., 0.]
4 for x, y in zip(rel_xs, rel_ys):
----> 5 nwbfile_read.add_electrode(rel_x=x, rel_y=y, enforce_unique_id=True, source="second", group=eg, location="unknown")
File /opt/conda/lib/python3.9/site-packages/hdmf/utils.py:668, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
666 def func_call(*args, **kwargs):
667 pargs = _check_args(args, kwargs)
--> 668 return func(args[0], **pargs)
File /opt/conda/lib/python3.9/site-packages/pynwb/file.py:717, in NWBFile.add_electrode(self, **kwargs)
714 else:
715 d.pop(col_name) # remove args from d if not set
--> 717 self.electrodes.add_row(**d)
File /opt/conda/lib/python3.9/site-packages/hdmf/utils.py:668, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
666 def func_call(*args, **kwargs):
667 pargs = _check_args(args, kwargs)
--> 668 return func(args[0], **pargs)
File /opt/conda/lib/python3.9/site-packages/hdmf/common/table.py:706, in DynamicTable.add_row(self, **kwargs)
704 if row_id in self.id:
705 raise ValueError("id %i already in the table" % row_id)
--> 706 self.id.append(row_id)
708 for colname, colnum in self.__colids.items():
709 if colname not in data:
File /opt/conda/lib/python3.9/site-packages/hdmf/container.py:876, in Data.append(self, arg)
874 def append(self, arg):
875 self._validate_new_data_element(arg)
--> 876 self.__data = append_data(self.__data, arg)
File /opt/conda/lib/python3.9/site-packages/hdmf/data_utils.py:32, in append_data(data, arg)
30 else:
31 msg = "Data cannot append to object of type '%s'" % type(data)
---> 32 raise ValueError(msg)
ValueError: Data cannot append to object of type '<class 'zarr.core.Array'>'
Operating System
Linux
Python Executable
Python
Python Version
3.9
Package Versions
pynwb 2.7.0 hdmf 3.13.0 hdmf-zarr 0.7.0
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
- [X] Have you checked the Contributing document?
- [X] Have you ensured this bug was not already reported?
Thanks for the issue and example, I believe this will most likely need to be addressed in HDMF so @mavaylon1 will open up a related issue to work on it there.
