pynwb
pynwb copied to clipboard
NeurodataWithoutBorders
'Cannot reassign parent to Container' error with current hdmf version
1) Bug
With the current version of HDMF on github an error Cannot reassign parent to Container is raised if I try to set the same electrodes DynamicTableRegion to more than one module. The following code runs without errors on current PyPi hdmf 1.0.4 version, but with the github version, it breaks:
(I wasn’t sure if it’s better to open this issues here or at hdmf, so sorry in advance if it should be put over there)
from datetime import datetime
from dateutil.tz import tzlocal
from pynwb import NWBFile, NWBHDF5IO, TimeSeries, ProcessingModule
from pynwb.ecephys import LFP, ElectricalSeries
import nwbext_ecog
import numpy as np
import os
# Creates file 1
nwb = NWBFile(session_description='session', identifier='1', session_start_time=datetime.now(tzlocal()))
# Add electrode groups and channels
nChannels = 10
dev0 = nwb.create_device(name='dev0')
elecs_group = nwb.create_electrode_group(name='electrodes', description='', location='ctx', device=dev0)
for i in np.arange(nChannels):
ii = float(i)
nwb.add_electrode(x=ii, y=ii, z=ii, imp=ii, location='', filtering='', group=elecs_group)
#Add signal
elecs_region = nwb.electrodes.create_region(name='electrodes',
region=np.arange(nChannels).tolist(),
description='')
X_data = np.zeros((nChannels,1000))
signal = ElectricalSeries(name='ElectricalSeries',
data=X_data,
electrodes=elecs_region,
rate=1.)
nwb.add_acquisition(signal)
# Extracellular group
ecephys_module = ProcessingModule(name='ecephys',
description='Extracellular electrophysiology data.')
nwb.add_processing_module(ecephys_module)
# Create LFP data
lfp = LFP()
lfp_ts = lfp.create_electrical_series(name='preprocessed',
data=X_data.T,
electrodes=nwb.acquisition['ElectricalSeries'].electrodes,
rate=1.,
description='')
ecephys_module.add_data_interface(lfp)
#Write file
with NWBHDF5IO('file_1.nwb', mode='w') as io:
io.write(nwb)
The error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-e1d9ac22c674> in <module>
40 electrodes=nwb.acquisition['ElectricalSeries'].electrodes,
41 rate=1.,
---> 42 description='')
43 ecephys_module.add_data_interface(lfp)
44
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
386 raise_from(ExceptionType(msg), None)
387
--> 388 return func(self, **parsed['args'])
389 else:
390 def func_call(*args, **kwargs):
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/pynwb/core.py in _func(self, **kwargs)
755 def _func(self, **kwargs):
756 cargs, ckwargs = fmt_docval_args(container_type.__init__, kwargs)
--> 757 ret = container_type(*cargs, **ckwargs)
758 getattr(self, add_name)(ret)
759 return ret
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
386 raise_from(ExceptionType(msg), None)
387
--> 388 return func(self, **parsed['args'])
389 else:
390 def func_call(*args, **kwargs):
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/pynwb/ecephys.py in __init__(self, **kwargs)
94 name, electrodes, data = popargs('name', 'electrodes', 'data', kwargs)
95 super(ElectricalSeries, self).__init__(name, data, 'volt', **kwargs)
---> 96 self.electrodes = electrodes
97
98
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/pynwb/core.py in nwbdi_setter(self, val)
266 val = [val]
267 for v in val:
--> 268 self.add_child(v)
269
270 ret.append(nwbdi_setter)
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
386 raise_from(ExceptionType(msg), None)
387
--> 388 return func(self, **parsed['args'])
389 else:
390 def func_call(*args, **kwargs):
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/container.py in add_child(self, **kwargs)
52 self.__children.append(child)
53 self.set_modified()
---> 54 child.parent = self
55 else:
56 warn('Cannot add None as child to a container %s' % self.name)
~/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/container.py in parent(self, parent_container)
95 if isinstance(self.parent, Container):
96 raise ValueError(('Cannot reassign parent to Container: %s. '
---> 97 'Parent is already: %s.' % (repr(self), repr(self.parent))))
98 else:
99 if parent_container is None:
ValueError: Cannot reassign parent to Container:
electrodes <class 'pynwb.core.DynamicTableRegion'>
Fields:
description:
table: electrodes <class 'pynwb.core.DynamicTable'>
. Parent is already:
ElectricalSeries <class 'pynwb.ecephys.ElectricalSeries'>
Fields:
comments: no comments
conversion: 1.0
data: [[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]
...
[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]]
description: no description
electrodes: electrodes <class 'pynwb.core.DynamicTableRegion'>
num_samples: 10
rate: 1.0
resolution: 0.0
starting_time: 0.0
unit: volt
Thanks for the report. While we sort out what the right behavior should be here, as a workaround, you can redefine elecs_region before you use it the second time.
elecs_region = nwb.electrodes.create_region(name='electrodes',
region=np.arange(nChannels).tolist(),
description='')
lfp_ts = lfp.create_electrical_series(name='preprocessed',
data=X_data.T,
electrodes=elecs_region,
rate=1.,
description='')
@rly thanks, that solves for this example. In the bigger project I'm working on, though, this workaround raises another error, at writing:
File "/home/tauffer/Github/ecogVIS/ecogvis/signal_processing/processing_data.py", line 224, in preprocess_raw_data
io.write(nwb)
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call
return func(self, **parsed['args'])
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 219, in write
call_docval_func(super(HDF5IO, self).write, kwargs)
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 281, in call_docval_func
return func(*fargs, **fkwargs)
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call
return func(self, **parsed['args'])
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/io.py", line 42, in write
self.write_builder(f_builder, **kwargs)
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call
return func(self, **parsed['args'])
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 446, in write_builder
self.write_group(self.__file, gbldr)
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call
return func(self, **parsed['args'])
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 599, in write_group
self.write_link(group, sub_builder)
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call
return func(self, **parsed['args'])
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 636, in write_link
parent[name] = link_obj
File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/h5py/_hl/group.py", line 392, in __setitem__
lcpl=lcpl, lapl=self._lapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5l.pyx", line 157, in h5py.h5l.LinkProxy.create_external
RuntimeError: Unable to create link (name already exists)
I still couldn't isolate the cause of it in order to give a simple reproducible code, but maybe you get an idea of what might be? I'll update it here if I can figure it out. Importantly, this only shows up with the github installation of hdmf, the version on pypi runs normally
Oh. It might be because you have two electrode table regions with the same name, and that is (currently) not allowed.
On Tue, Jun 25, 2019 at 10:38 AM Luiz Tauffer [email protected] wrote:
@rly https://github.com/rly thanks, that solves for this example. In the bigger project I'm working on, though, this workaround raises another error:
File "/home/tauffer/Github/ecogVIS/ecogvis/signal_processing/processing_data.py", line 224, in preprocess_raw_data io.write(nwb) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call return func(self, **parsed['args']) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 219, in write call_docval_func(super(HDF5IO, self).write, kwargs) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 281, in call_docval_func return func(*fargs, **fkwargs) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call return func(self, **parsed['args']) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/io.py", line 42, in write self.write_builder(f_builder, **kwargs) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call return func(self, **parsed['args']) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 446, in write_builder self.write_group(self.__file, gbldr) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call return func(self, **parsed['args']) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 599, in write_group self.write_link(group, sub_builder) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py", line 388, in func_call return func(self, **parsed['args']) File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py", line 636, in write_link parent[name] = link_obj File "/home/tauffer/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/h5py/_hl/group.py", line 392, in setitem lcpl=lcpl, lapl=self._lapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5l.pyx", line 157, in h5py.h5l.LinkProxy.create_externalRuntimeError: Unable to create link (name already exists)
I still couldn't isolate the cause of it in order to give a simple reproducible code, but maybe you get an idea of what might be? I'll update it here if I can figure it out. Importantly, this only shows up with the github installation of hdmf, the version on pypi runs normally
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NeurodataWithoutBorders/pynwb/issues/982?email_source=notifications&email_token=AACLXNO3W5DEO4JFW6MFJD3P4JJZPA5CNFSM4H3KAVVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYRATOA#issuecomment-505547192, or mute the thread https://github.com/notifications/unsubscribe-auth/AACLXNIIMOCS3OIKDC3CNPLP4JJZPANCNFSM4H3KAVVA .
-- Ryan Ly Scientific Data Engineer Lawrence Berkeley National Laboratory 1 Cyclotron Road Mail Stop 59R4104 Berkeley, CA 94720 Email: [email protected] Pronouns: he/him
Ok, so the problem seems to be incompatibility between NWB files created with previous versions of hdmf and the new one (github version). I created a new NWB file with the same data as my old file (older hdmf) and this new file can be written just fine now.
I've updated hdmf to 1.0.5. That should fix the problem. Please re-open and comment if it does not.
@rly it dd not solve the problem here, getting the same error as before
@luiztauffer sorry, can you clarify the current problem? Are you still getting a RuntimeError: Unable to create link (name already exists) ? I am trying to mock together something that reproduces your error based on what I see in https://github.com/luiztauffer/ecogVIS/blob/master/ecogvis/signal_processing/processing_data.py but have so far not been successful.