gh-69605: Check for already imported modules in PyREPL module completion
This PR handles an edge-case when suggesting module imports in PyREPL (see this comment):
If a module is already imported, the import machinery will only get submodules from it, even if a module of the same name is higher in the path. This can happen if eg. sys.path has been updated since the module was originaly imported.
The easiest way to reproduce this issue (and how I stumbled on it) is with stdlib modules imported during interpreter startup (before site customisation, so ignoring potential shadowing):
% mkdir collections
% touch collections/__init__.py collections/foo.py
% python
Python 3.14.0rc2 [...]
>>> from collections import <TAB>
>>> from collections import foo
Traceback (most recent call last):
File "<python-input-0>", line 1, in <module>
from collections import foo
ImportError: cannot import name 'foo' from 'collections' (/Users/loic/cpython/Lib/collections/__init__.py)
>>>
This PR add a special case in _pyrepl.ModuleCompleter._find_modules that
- check if the module to complete imports from is already imported
- if yes, search for imports from the imported module location only, instead of using the global cache
It was a little tricky to get right, I'm not sure about everything 😅 but it pass all tests I could think off!
cc @tomasr8
- Issue: gh-69605
Ok, one of the new test fails on Windows x64/32 (but pass on arm)
I just setup and built Python on a Win 11 x64 to try to reproduce, the test pass 😵💫 (and the fix works)
If anyone has an idea of what may interfere? (Python 3.15.0a0 (main, Sep 30 2025, 23:36:53) [MSC v.1944 64 bit (AMD64)] on win32)
I'm going to convert it into a draft until you deem this PR ready. That will also reduce the probability of someone clicking on the PR to review it (like me...) while it's not yet ready for.
I'm going to convert it into a draft until you deem this PR ready. That will also reduce the probability of someone clicking on the PR to review it (like me...) while it's not yet ready for.
Thanks, I just fixed the failing test, this should be ready for review!
Sorry again for the noise, I was dealing with what turned out to be a importer cache issue, which I couldn't reproduce outside of the CI buildbots 😓
[edit] I just noticed I can convert to draft / mark ready myself, will do it next times!
This is good, confirmed on macOS and Windows it's doing the right thing!
:warning::warning::warning: Buildbot failure :warning::warning::warning:
Hi! The buildbot s390x RHEL9 LTO + PGO 3.x (tier-3) has failed when building commit bfac54d8611f60c5b19640c410cea696a81457bb.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1578/builds/3864) and take a look at the build logs.
- Check if the failure is related to this commit (bfac54d8611f60c5b19640c410cea696a81457bb) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/1578/builds/3864
Failed tests:
- test_pyrepl
Failed subtests:
- test_already_imported_module_without_origin_or_spec - test.test_pyrepl.test_pyrepl.TestPyReplModuleCompleter.test_already_imported_module_without_origin_or_spec
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
module = importlib.import_module(mod)
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/importlib/__init__.py", line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'no_spec'
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
module = importlib.import_module(mod)
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/importlib/__init__.py", line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'not_has_location'
The buildbot failure is related. I'm on it.
:warning::warning::warning: Buildbot failure :warning::warning::warning:
Hi! The buildbot s390x RHEL9 LTO 3.x (tier-3) has failed when building commit bfac54d8611f60c5b19640c410cea696a81457bb.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1587/builds/3876) and take a look at the build logs.
- Check if the failure is related to this commit (bfac54d8611f60c5b19640c410cea696a81457bb) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/1587/builds/3876
Failed tests:
- test_pyrepl
Failed subtests:
- test_already_imported_module_without_origin_or_spec - test.test_pyrepl.test_pyrepl.TestPyReplModuleCompleter.test_already_imported_module_without_origin_or_spec
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
module = importlib.import_module(mod)
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/importlib/__init__.py", line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'not_has_location'
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
module = importlib.import_module(mod)
File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/importlib/__init__.py", line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'no_spec'
I suspect this is about file system timestamp resolution. Lemme attempt a fix.
@ambv Shall we backport this PR (& and the buildbot fix)? I think this can be considered a bugfix.
Yup, gh-143438.