`PathFinder.find_spec()` can raise bare `KeyError` when `path=None`

[jacobtylerwalls]

Bug report For the following tree, where ./b is not a package, PathFinder.find_spec("a.b") raises an undocumented KeyError:

.
├── a
│   ├── b.py
└── b

>>> from importlib.machinery import PathFinder
>>> PathFinder.find_spec("a.b")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap_external>", line 1439, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1218, in __init__
  File "<frozen importlib._bootstrap_external>", line 1233, in _get_parent_path
KeyError: 'a'

Works as expected if a is provided as the path argument:

>>> PathFinder.find_spec("a.b", path=["a"])
ModuleSpec(name='a.b', loader=<_frozen_importlib_external.SourceFileLoader object at 0x100d37d30>, origin='/Users/myuser/a/b.py')

Your environment

Python 3.10.2 (v3.10.2:a58ebcc701, Jan 13 2022, 14:50:16) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin

Also reproduced on 3.8 and 3.12.0a0

Linked PRs

• gh-98100

[jacobtylerwalls]

I see from #89754 that this is invalid/poor usage if you know in advance that there's a resource under a/, but probably ~None is the better outcome?~ the correct outcome would be, I think, what you get when the a/b.py file is not present, namely:

ModuleSpec(name='b', loader=None, submodule_search_locations=_NamespacePath(['<CWD>/b']))

[serhiy-storchaka]

I do not think that #98100 is the right solution. It gets rid of KeyError, but returns a wrong result.

[jacobtylerwalls]

Thanks for taking a look @serhiy-storchaka.

The result I get with #98100 is the result when the a/b.py file is not present, namely the path to the /b directory. What would be a more correct result?

(I don't think we want to return the path to a/b.py without the user providing a path arg based on https://github.com/python/cpython/issues/89754#issuecomment-1093935005.)

[serhiy-storchaka]

I think that it should be the same as if the /b directory did not exist.

[brettcannon]

I think that it should be the same as if the /b directory did not exist.

I agree if Serhiy means it should raise ModuleNotFoundError. The error stems from trying to import a submodule w/o the parent module being imported. Directories have always taken precedence, so I don't think this should fall back to the module.

[jacobtylerwalls]

Without the name collision, trying to find the submodule without a parent import or a path arg just returns None:

.
├── a
│   ├── b.py
└── c

>>> from importlib.machinery import PathFinder
>>> PathFinder.find_spec("a.b") is None
>>> True

[brettcannon]

The fix went into 3.15, but I don't want to backport as it is technically a backwards-incompatible change.