core icon indicating copy to clipboard operation
core copied to clipboard

regression in processor.resolve_resource: module may be a namespace package only

Open bertsky opened this issue 3 years ago • 1 comments
trafficstars

Since #904 we look up … https://github.com/OCR-D/core/blob/71d295ac1fccbeb4164e230bd584e1920b9ab3c8/ocrd/ocrd/processor/base.py#L246-L250 … to determine the moduledir location.

But that fails if the processor module itself is distributed via PEP-420 namespace packages (so the top level does not contain any resources).

For example, in case of ocrd-eynollah-segment, we have qurator.eynollah as the actual processor module, but qurator itself is empty:

  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/qurator/eynollah/processor.py", line 47, in process
    'dir_models': self.resolve_resource(self.parameter['models']),
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/ocrd/processor/base.py", line 219, in resolve_resource
    cwd=cwd, moduled=self.moduledir)
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/ocrd/processor/base.py", line 257, in moduledir
    return resource_filename(self.module, '')
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1154, in resource_filename
    return get_provider(package_or_requirement).get_resource_filename(
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 361, in get_provider
    return _find_adapter(_provider_factories, loader)(module)
  File "/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1403, in __init__
    self.module_path = os.path.dirname(getattr(module, '__file__', ''))
  File "/usr/lib/python3.7/posixpath.py", line 156, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

I don't know what's the correct way to deal with this, but it seems to suffice checking that a .__file__ exists at the module level – for namespaces this is None.

bertsky avatar Oct 05 '22 11:10 bertsky

I don't know what's the correct way to deal with this, but it seems to suffice checking that a .__file__ exists at the module level – for namespaces this is None.

Judging by the documentation of __file__ and __package__, I do think this is the test we have to make: traverse the module path from top to current and stop as soon as we find a package that has_location.

bertsky avatar Oct 05 '22 12:10 bertsky