pynwb
pynwb copied to clipboard
NeurodataWithoutBorders
[Bug]: 'Core' not a namespace
What happened?
I attempted to run PyNWB.NWBHDF5IO in a Jupyter notebook while remotely accessing nwb files from DANDI. The code that yields an error is code from the docs. This snippet below
import fsspec
import pynwb
import h5py
from fsspec.implementations.cached import CachingFileSystem
# first, create a virtual filesystem based on the http protocol and use
# caching to save accessed data to RAM.
fs = CachingFileSystem(
fs=fsspec.filesystem("http"),
cache_storage="nwb-cache", # Local folder for the cache
)
# next, open the file
with fs.open(file_url, "rb") as f:
with h5py.File(f) as file:
with pynwb.NWBHDF5IO(file=file, mode='r', load_namespaces=True) as io:
nwbfile = io.read()
And then I received an error; 'core not a namespace'
Steps to Reproduce
- Clone and and install PyNWB from the github repo as described here
- Run
pip install dandiandpip install fsspec requests aiohttp - Run a jupyter notebook locally on your machine.
- Include and run a cell with the following snippet
from dandi import dandiapi
client = dandiapi.DandiAPIClient()
my_dandiset = client.get_dandiset("000004")
file = my_dandiset.get_asset_by_path("sub-P18HMH/sub-P18HMH_ses-20080601_ecephys+image.nwb")
file_url = file.get_content_url(follow_redirects=1, strip_query=True)
- Then include and run the snippet which caused the error shown in the above section
Traceback
The traceback is as follows
KeyError Traceback (most recent call last)
Cell In [6], line 2
1 import fsspec
----> 2 import pynwb
3 import h5py
4 from fsspec.implementations.cached import CachingFileSystem
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\__init__.py:294
290 kwargs['container'] = nwbfile
291 super().export(**kwargs)
--> 294 from . import io as __io # noqa: F401,E402
295 from .core import NWBContainer, NWBData # noqa: F401,E402
296 from .base import TimeSeries, ProcessingModule # noqa: F401,E402
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\io\__init__.py:1
----> 1 from . import base as __base
2 from . import core as __core
3 from . import file as __file
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\io\base.py:3
1 from hdmf.build import LinkBuilder
----> 3 from .core import NWBContainerMapper
4 from .. import register_map
5 from ..base import TimeSeries, ProcessingModule
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\io\core.py:9
5 from hdmf.build import BuildManager
7 from .. import register_map
----> 9 from pynwb.file import NWBFile
10 from pynwb.core import NWBData, NWBContainer, ScratchData
11 from pynwb.misc import Units
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\file.py:14
11 from hdmf.utils import docval, getargs, get_docval, popargs, popargs_to_dict, AllowPositional
13 from . import register_class, CORE_NAMESPACE
---> 14 from .base import TimeSeries, ProcessingModule
15 from .device import Device
16 from .epoch import TimeIntervals
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\base.py:12
9 from hdmf.utils import get_data_shape
11 from . import register_class, CORE_NAMESPACE
---> 12 from .core import NWBDataInterface, MultiContainerInterface, NWBData
15 @register_class('ProcessingModule', CORE_NAMESPACE)
16 class ProcessingModule(MultiContainerInterface):
17 """ Processing module. This is a container for one or more containers
18 that provide data at intermediate levels of analysis
19
20 ProcessingModules should be created through calls to NWB.create_module().
21 They should not be instantiated directly
22 """
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\core.py:51
46 raise ValueError(error_msg)
47 warn(error_msg)
50 @register_class('NWBContainer', CORE_NAMESPACE)
---> 51 class NWBContainer(NWBMixin, Container):
53 _fieldsname = '__nwbfields__'
55 __nwbfields__ = tuple()
File c:\users\carter.peene\desktop\projects\pynwb\src\pynwb\__init__.py:130, in register_class.<locals>._dec(cls)
129 def _dec(cls):
--> 130 __TYPE_MAP.register_container_type(namespace, neurodata_type, cls)
131 return cls
File ~\AppData\Local\Programs\Python\Python39\lib\site-packages\hdmf\utils.py:645, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
643 def func_call(*args, **kwargs):
644 pargs = _check_args(args, kwargs)
--> 645 return func(args[0], **pargs)
File ~\AppData\Local\Programs\Python\Python39\lib\site-packages\hdmf\build\manager.py:724, in TypeMap.register_container_type(self, **kwargs)
722 ''' Map a container class to a data_type '''
723 namespace, data_type, container_cls = getargs('namespace', 'data_type', 'container_cls', kwargs)
--> 724 spec = self.__ns_catalog.get_spec(namespace, data_type) # make sure the spec exists
725 self.__container_types.setdefault(namespace, dict())
726 self.__container_types[namespace][data_type] = container_cls
File ~\AppData\Local\Programs\Python\Python39\lib\site-packages\hdmf\utils.py:645, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
643 def func_call(*args, **kwargs):
644 pargs = _check_args(args, kwargs)
--> 645 return func(args[0], **pargs)
File ~\AppData\Local\Programs\Python\Python39\lib\site-packages\hdmf\spec\namespace.py:315, in NamespaceCatalog.get_spec(self, **kwargs)
313 namespace, data_type = getargs('namespace', 'data_type', kwargs)
314 if namespace not in self.__namespaces:
--> 315 raise KeyError("'%s' not a namespace" % namespace)
316 return self.__namespaces[namespace].get_spec(data_type)
KeyError: "'core' not a namespace"
Operating System
Windows
Python Executable
Python
Python Version
3.9
Package Versions
aiohttp==3.8.3 aiosignal==1.2.0 anyio==3.6.1 appdirs==1.4.4 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 arrow==1.2.3 asciitree==0.3.3 asttokens==2.0.8 async-timeout==4.0.2 attrs==22.1.0 backcall==0.2.0 beautifulsoup4==4.11.1 bidsschematools==0.5.1 bleach==5.0.1 ccfwidget==0.5.3 certifi==2022.9.14 cffi==1.15.1 charset-normalizer==2.1.1 ci-info==0.3.0 click==8.1.3 click-didyoumean==0.3.0 cloudpickle==2.2.0 colorama==0.4.5 colorcet==3.0.0 contourpy==1.0.5 cycler==0.11.0 dandi==0.46.3 dandischema==0.7.1 dask==2022.9.1 debugpy==1.6.3 decorator==5.1.1 defusedxml==0.7.1 dnspython==2.2.1 email-validator==1.3.0 entrypoints==0.4 etelemetry==0.3.0 executing==1.0.0 fasteners==0.18 fastjsonschema==2.16.2 fonttools==4.37.3 fqdn==1.5.1 frozenlist==1.3.1 fscacher==0.2.0 fsspec==2022.8.2 h5py==3.7.0 hdmf==3.4.2 humanize==4.4.0 idna==3.4 imageio==2.22.0 importlib-metadata==4.12.0 interleave==0.2.1 ipydatawidgets==4.3.2 ipykernel==6.15.3 ipympl==0.9.2 ipysheet==0.5.0 ipython==8.5.0 ipython-genutils==0.2.0 ipytree==0.2.2 ipyvolume==0.5.2 ipywebrtc==0.6.0 ipywidgets==7.7.2 isoduration==20.11.0 itk-core==5.2.1.post1 itk-filtering==5.2.1.post1 itk-meshtopolydata==0.8.1 itk-numerics==5.2.1.post1 itkwidgets==0.32.3 jaraco.classes==3.2.2 jedi==0.18.1 Jinja2==3.1.2 JIT==0.0.1 joblib==1.2.0 jsonpointer==2.3 jsonschema==4.16.0 jupyter-core==4.11.1 jupyter-server==1.18.1 jupyter_client==7.3.5 jupyterlab-pygments==0.2.2 jupyterlab-widgets==1.1.1 keyring==23.9.3 keyrings.alt==4.2.0 kiwisolver==1.4.4 llvmlite==0.39.1 locket==1.0.0 lxml==4.9.1 MarkupSafe==2.1.1 matplotlib==3.6.0 matplotlib-inline==0.1.6 mistune==2.0.4 more-itertools==8.14.0 multidict==6.0.2 natsort==8.2.0 nbclient==0.6.8 nbconvert==7.0.0 nbformat==5.6.0 ndx-grayscalevolume==0.0.2 ndx-icephys-meta==0.1.0 ndx-spectrum==0.2.2 nest-asyncio==1.5.5 networkx==2.8.6 notebook==6.4.12 numba==0.56.2 numcodecs==0.10.2 numpy==1.21.5 nwbinspector==0.4.13 nwbwidgets==0.8.0 opencv-python==4.6.0.66 packaging==21.3 pandas==1.3.5 pandocfilters==1.5.0 param==1.12.2 parso==0.8.3 partd==1.3.0 pickleshare==0.7.5 Pillow==9.2.0 plotly==5.10.0 prometheus-client==0.14.1 prompt-toolkit==3.0.31 psutil==5.9.2 pure-eval==0.2.2 pycparser==2.21 pycryptodomex==3.15.0 pyct==0.4.8 pydantic==1.10.2 Pygments==2.13.0 -e git+https://github.com/NeurodataWithoutBorders/pynwb.git@6e1f131c32ead004271d143d00c6fee05dbd9ef6#egg=pynwb pyout==0.7.2 pyparsing==3.0.9 pyrsistent==0.18.1 python-dateutil==2.8.2 pythreejs==2.4.1 pytz==2022.2.1 PyWavelets==1.4.1 pywin32==304 pywin32-ctypes==0.2.0 pywinpty==2.0.8 PyYAML==6.0 pyzmq==24.0.1 requests==2.28.1 rfc3339-validator==0.1.4 rfc3987==1.3.8 ruamel.yaml==0.17.21 ruamel.yaml.clib==0.2.6 scikit-image==0.19.3 scikit-learn==1.1.2 scipy==1.9.1 semantic-version==2.10.0 Send2Trash==1.8.0 six==1.16.0 sniffio==1.3.0 soupsieve==2.3.2.post1 stack-data==0.5.0 tenacity==8.1.0 terminado==0.15.0 threadpoolctl==3.1.0 tifffile==2022.8.12 tinycss2==1.1.1 toolz==0.12.0 tornado==6.2 tqdm==4.64.1 traitlets==5.4.0 traittypes==0.2.1 trimesh==3.15.2 typing_extensions==4.3.0 uri-template==1.2.0 urllib3==1.26.12 util-colleenjg==0.0.1 wcwidth==0.2.5 webcolors==1.12 webencodings==0.5.1 websocket-client==1.4.1 widgetsnbextension==3.6.1 xarray==2022.6.0 yarl==1.8.1 zarr==2.13.0 zipp==3.8.1 zstandard==0.18.0
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
- [X] Have you checked the Contributing document?
- [X] Have you ensured this bug was not already reported?
Ah, no its not! I neglected to include that. It's dandiset_id 000004 and file sub-P18HMH/sub-P18HMH_ses-20080601_ecephys+image.nwb. Will edit my instruction snippet to include this.
@rcpeene
Can you please install pynwb using these instructions? You need to do a special command that includes submodules. It should work once you do that
Excellent. I think that fixed this issue, but I can't say verify this for certain because I am getting a new error now with the same code. Here's the traceback.
TypeError Traceback (most recent call last)
Cell In [6], line 16
14 with fs.open(file_url, "rb") as f:
15 with h5py.File(f) as file:
---> 16 with pynwb.NWBHDF5IO(file=file, mode='r', load_namespaces=True) as io:
17 nwbfile = io.read()
File ~\AppData\Local\Programs\Python\Python39\lib\site-packages\hdmf\utils.py:644, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
643 def func_call(*args, **kwargs):
--> 644 pargs = _check_args(args, kwargs)
645 return func(args[0], **pargs)
File ~\AppData\Local\Programs\Python\Python39\lib\site-packages\hdmf\utils.py:637, in docval.<locals>.dec.<locals>._check_args(args, kwargs)
635 if parse_err:
636 msg = '%s: %s' % (func.__qualname__, ', '.join(parse_err))
--> 637 raise ExceptionType(msg)
639 return parsed['args']
TypeError: NWBHDF5IO.__init__: missing argument 'path'
Is this me misusing the example code or a bug with the package?
Did you do this?
$ git clone --recurse-submodules [email protected]:NeurodataWithoutBorders/pynwb.git
$ cd pynwb
$ pip install -r requirements.txt
$ pip install -e .
TypeError: NWBHDF5IO.init: missing argument 'path'
I think you may need to update the version of HDMF you are using
I think the install step was interrupted for some reason and I missed it. These fixed the problem. Thanks!
As a point of curiosity, is installing via the github repo the only way to prevent this error? Are there resulting packaging issues when installing via pip? This is relevant for my usage of this tool in a Jupyter notebook where I'd like to try and make it's usage environment-agnostic.
As a point of curiosity, is installing via the github repo the only way to prevent this error? Are there resulting packaging issues when installing via pip?
Installing via pip is fine
Seems as though the NWBHDF5IO.__init__: missing argument 'path' arises again when testing it from a pip installation.
Seems as though the
NWBHDF5IO.__init__: missing argument 'path'arises again when testing it from a pip installation.
Can you try again. There is a new release of HDMF which I believe includes the fix for this
The new HDMF release does not fix this. But the latest dev branch of PyNWB does fix this. You have to install pynwb from source using the git clone command that @bendichter included above.