cpython icon indicating copy to clipboard operation
cpython copied to clipboard

gh-69605: Check for already imported modules in PyREPL module completion

Open loic-simon opened this issue 3 months ago • 4 comments

This PR handles an edge-case when suggesting module imports in PyREPL (see this comment):

If a module is already imported, the import machinery will only get submodules from it, even if a module of the same name is higher in the path. This can happen if eg. sys.path has been updated since the module was originaly imported.

The easiest way to reproduce this issue (and how I stumbled on it) is with stdlib modules imported during interpreter startup (before site customisation, so ignoring potential shadowing):

% mkdir collections
% touch collections/__init__.py collections/foo.py
% python
Python 3.14.0rc2 [...]
>>> from collections import <TAB>
>>> from collections import foo
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    from collections import foo
ImportError: cannot import name 'foo' from 'collections' (/Users/loic/cpython/Lib/collections/__init__.py)
>>>

This PR add a special case in _pyrepl.ModuleCompleter._find_modules that

  • check if the module to complete imports from is already imported
  • if yes, search for imports from the imported module location only, instead of using the global cache

It was a little tricky to get right, I'm not sure about everything 😅 but it pass all tests I could think off!

cc @tomasr8

  • Issue: gh-69605

loic-simon avatar Sep 30 '25 20:09 loic-simon

Ok, one of the new test fails on Windows x64/32 (but pass on arm)

I just setup and built Python on a Win 11 x64 to try to reproduce, the test pass 😵‍💫 (and the fix works) If anyone has an idea of what may interfere? (Python 3.15.0a0 (main, Sep 30 2025, 23:36:53) [MSC v.1944 64 bit (AMD64)] on win32)

loic-simon avatar Sep 30 '25 21:09 loic-simon

All commit authors signed the Contributor License Agreement.

CLA signed

python-cla-bot[bot] avatar Oct 01 '25 23:10 python-cla-bot[bot]

I'm going to convert it into a draft until you deem this PR ready. That will also reduce the probability of someone clicking on the PR to review it (like me...) while it's not yet ready for.

picnixz avatar Oct 03 '25 13:10 picnixz

I'm going to convert it into a draft until you deem this PR ready. That will also reduce the probability of someone clicking on the PR to review it (like me...) while it's not yet ready for.

Thanks, I just fixed the failing test, this should be ready for review!

Sorry again for the noise, I was dealing with what turned out to be a importer cache issue, which I couldn't reproduce outside of the CI buildbots 😓

[edit] I just noticed I can convert to draft / mark ready myself, will do it next times!

loic-simon avatar Oct 03 '25 14:10 loic-simon

This is good, confirmed on macOS and Windows it's doing the right thing!

ambv avatar Jan 05 '26 15:01 ambv

:warning::warning::warning: Buildbot failure :warning::warning::warning:

Hi! The buildbot s390x RHEL9 LTO + PGO 3.x (tier-3) has failed when building commit bfac54d8611f60c5b19640c410cea696a81457bb.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1578/builds/3864) and take a look at the build logs.
  4. Check if the failure is related to this commit (bfac54d8611f60c5b19640c410cea696a81457bb) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1578/builds/3864

Failed tests:

  • test_pyrepl

Failed subtests:

  • test_already_imported_module_without_origin_or_spec - test.test_pyrepl.test_pyrepl.TestPyReplModuleCompleter.test_already_imported_module_without_origin_or_spec

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
    module = importlib.import_module(mod)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'no_spec'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
    module = importlib.import_module(mod)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'not_has_location'

bedevere-bot avatar Jan 05 '26 15:01 bedevere-bot

The buildbot failure is related. I'm on it.

ambv avatar Jan 05 '26 15:01 ambv

:warning::warning::warning: Buildbot failure :warning::warning::warning:

Hi! The buildbot s390x RHEL9 LTO 3.x (tier-3) has failed when building commit bfac54d8611f60c5b19640c410cea696a81457bb.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1587/builds/3876) and take a look at the build logs.
  4. Check if the failure is related to this commit (bfac54d8611f60c5b19640c410cea696a81457bb) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1587/builds/3876

Failed tests:

  • test_pyrepl

Failed subtests:

  • test_already_imported_module_without_origin_or_spec - test.test_pyrepl.test_pyrepl.TestPyReplModuleCompleter.test_already_imported_module_without_origin_or_spec

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
    module = importlib.import_module(mod)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'not_has_location'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1195, in test_already_imported_module_without_origin_or_spec
    module = importlib.import_module(mod)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.lto/build/Lib/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1303, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1276, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1240, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'no_spec'

bedevere-bot avatar Jan 05 '26 15:01 bedevere-bot

I suspect this is about file system timestamp resolution. Lemme attempt a fix.

ambv avatar Jan 05 '26 16:01 ambv

@ambv Shall we backport this PR (& and the buildbot fix)? I think this can be considered a bugfix.

tomasr8 avatar Jan 05 '26 17:01 tomasr8

Yup, gh-143438.

ambv avatar Jan 05 '26 17:01 ambv