pynwb
pynwb copied to clipboard
NeurodataWithoutBorders
[Bug]: `pynwb.validate` can not check with `NWBZarrIO`
What happened?
The pynwb.validate is designed to operate on both NWBHDF5IO or NWBZarrIO. But I found the current implementation cannot work properly when checking a Zarr-compressed NWB file. It seems to _validate_single_file incorrectly invoked the API of NWBZarrIO.
The issue is similar to https://github.com/hdmf-dev/hdmf-zarr/issues/293, which also reports a validation error when checking a zarr-compressed nwb file. But its traceback seems to be different.
Steps to Reproduce
Create a zarr-compressed NWB file.
from hdmf_zarr import NWBZarrIO
with NWBZarrIO(output_path, 'w') as io:
io.write(nwbfile)
Make sure that the file can be read by NWBZarrIO
print(NWBZarrIO.can_read(output_path))
should print True
Then, validate it:
import pynwb
pynwb.validate(path=output_path)
There should be errors, which is further attached below.
Traceback
The traceback varies depending on the configuration of use_cached_namespaces.
use_cached_namespaces=True (Default)
KeyError Traceback (most recent call last)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/spec/namespace.py:466, in NamespaceCatalog.__load_namespace(self, namespace, reader, resolve)
465 try:
--> 466 inc_ns = self.get_namespace(s['namespace'])
467 except KeyError as e:
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/utils.py:578, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
577 pargs = _check_args(args, kwargs)
--> 578 return func(args[0], **pargs)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/spec/namespace.py:317, in NamespaceCatalog.get_namespace(self, **kwargs)
316 if ret is None:
--> 317 raise KeyError("'%s' not a namespace" % name)
318 return ret
KeyError: "'core' not a namespace"
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
Cell In[9], line 3
1 import pynwb
----> 3 pynwb.validate(path=output_path, use_cached_namespaces=True)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/utils.py:582, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
580 def func_call(*args, **kwargs):
581 pargs = _check_args(args, kwargs)
--> 582 return func(**pargs)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/pynwb/validation.py:163, in validate(**kwargs)
161 validation_errors += _validate_single_file(path=p, **kwargs)
162 else:
--> 163 validation_errors = _validate_single_file(path=path, **kwargs)
165 return validation_errors
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/pynwb/validation.py:181, in _validate_single_file(**kwargs)
0 <Error retrieving source code with stack_data see ipython/ipython#13598>
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/pynwb/validation.py:70, in get_cached_namespaces_to_validate(path, driver, aws_region, io)
68 from pynwb import _get_backend
69 backend_io = _get_backend(path, method=driver)
---> 70 namespace_dependencies = backend_io.load_namespaces(namespace_catalog=catalog,
71 path=path,
72 driver=driver,
73 aws_region=aws_region)
75 # Determine which namespaces are the most specific (i.e. extensions) and validate against those
76 candidate_namespaces = set(namespace_dependencies.keys())
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/utils.py:578, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
576 def func_call(*args, **kwargs):
577 pargs = _check_args(args, kwargs)
--> 578 return func(args[0], **pargs)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf_zarr/backend.py:285, in ZarrIO.load_namespaces(cls, namespace_catalog, path, storage_options, namespaces)
283 ns_group = ns_group[latest_version]
284 reader = ZarrSpecReader(ns_group)
--> 285 namespace_catalog.load_namespaces("namespace", reader=reader)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/utils.py:578, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
576 def func_call(*args, **kwargs):
577 pargs = _check_args(args, kwargs)
--> 578 return func(args[0], **pargs)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/spec/namespace.py:578, in NamespaceCatalog.load_namespaces(self, **kwargs)
576 # now load specs into namespace
577 for ns in to_load:
--> 578 ret[ns['name']] = self.__load_namespace(ns, r, resolve=resolve)
579 self.__included_specs[ns_path_key] = ret
581 # warn if there are any ignored namespaces
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/spec/namespace.py:468, in NamespaceCatalog.__load_namespace(self, namespace, reader, resolve)
466 inc_ns = self.get_namespace(s['namespace'])
467 except KeyError as e:
--> 468 raise ValueError("Could not load namespace '%s'" % s['namespace']) from e
469 if types_to_load is None:
470 types_to_load = inc_ns.get_registered_types() # load all types in namespace
ValueError: Could not load namespace 'core'
use_cached_namespaces=False
TypeError Traceback (most recent call last)
Cell In[10], line 3
1 import pynwb
----> 3 pynwb.validate(path=output_path, use_cached_namespaces=False)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/utils.py:582, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
580 def func_call(*args, **kwargs):
581 pargs = _check_args(args, kwargs)
--> 582 return func(**pargs)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/pynwb/validation.py:163, in validate(**kwargs)
161 validation_errors += _validate_single_file(path=p, **kwargs)
162 else:
--> 163 validation_errors = _validate_single_file(path=path, **kwargs)
165 return validation_errors
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/pynwb/validation.py:202, in _validate_single_file(**kwargs)
200 from pynwb import _get_backend
201 backend_io = _get_backend(path, method=driver)
--> 202 io = backend_io(**io_kwargs)
204 # check namespaces are accurate
205 if namespace is not None:
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/utils.py:577, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
576 def func_call(*args, **kwargs):
--> 577 pargs = _check_args(args, kwargs)
578 return func(args[0], **pargs)
File ~/miniforge3/envs/dbci/lib/python3.13/site-packages/hdmf/utils.py:570, in docval.<locals>.dec.<locals>._check_args(args, kwargs)
568 if parse_err:
569 msg = '%s: %s' % (func.__qualname__, ', '.join(parse_err))
--> 570 raise ExceptionType(msg)
572 return parsed['args']
TypeError: NWBZarrIO.__init__: unrecognized argument: 'driver'
Operating System
Linux
Python Executable
Conda
Python Version
3.13
Package Versions
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
- [x] Have you checked the Contributing document?
- [x] Have you ensured this bug was not already reported?
Thanks for reporting this issue and for the detailed breakdown @MGAMZ. Validation of NWB Zarr files is not yet fully supported, but is something we are working on! There are a couple of remaining updates needed in hdmf-zarr.
I will let you know here when we have an update.
