Local import inside conda editable package doesn't work.

[elashrry]

Environment data

• Language Server version: 2022.9.21
• OS and version: darwin arm64
• Python version (and distribution if applicable, e.g. Anaconda): Python3.9 via anaconda
• python.analysis.indexing: null
• python.analysis.typeCheckingMode: off

Repro Steps

1. open new working space test in VSCode and have the following folders structure:

test/
    folder1/
        module.py
        test.py
        test.ipynb
    folder2/
        subfolder/
            module.py
            test.py
            test.ipynb

The contents of folder1/module.py and folder2/subfolder/module.py are:

def func():
    """
    This is a function
    """
    print("Hello World")

and all other test.py and test.ipynb have one line

from module import func

2. create a conda environment with Python3.9 (or any other version)
3. activate it and use it as python interpreter for the working space.
4. verify all local imports are resolved correctly by Pylance.
5. add folder2 as an editable package via conda develope ./folder2 from the root directory of the working space.
6. Now local import inside folder2/subfolder does not work.

Expected behavior

Local import inside conda editable package work normally.

Actual behavior

once you add a folder to conda using the command conda develop directory/path, the local import breaks inside this package.

Note

If I use explicit relative import, Pylance works normally inside `folder2/subfolder` but python will not work with explicit relative import in the running scripts or notebooks.

Logs

Pylance_logs.txt

[Jacques-Peeters]

Experiencing a similar issue on Podman. Very annoying.

[debonte]

Here's a repro that does not involve conda:

<workspace-root>\folder2\subfolder\module.py:

def func():
    """
    This is a function
    """
    print("Hello World")

<workspace-root>\folder2\subfolder\test.py:

from module import func

The import above works fine until you add the following as a .pth file in the site-packages directory, which is what the conda develop ./folder2 command does:

<absolute-path-to-workspace-root>\folder2

Once the .pth file is present, ParentDirectoryCache.checkValidPath will return false. There's a comment there that says:

if (this._libPathCache.some((p) => sourceFilePath.startsWith(p))) {
    // Make sure it is not lib folders under user code root.
    // ex) .venv folder
    return false;
}

And now the import is flagged as an error: Import "module" could not be resolved

@heejaechang, @erictraut, it's not clear to me if this is a bug or by design. Having a .pth file that points to a location within the workspace seems a little weird?

[debonte]

Btw, I also noticed that if the .pth points to <absolute-path-to-workspace-root>\folder2\subfolder, then the import works. It seems strange to me that the .pth pointing to folder2 causes an error but pointing to folder2\subfolder does not.

[erictraut]

@jakebailey added the code that implements the heuristics for module import resolution within the local project. Maybe he remembers why he added that condition.

[jakebailey]

If this is the "go up from the current module" heuristic, I didn't implement it; I did push the code out to pyright, though, if you're going off of the pyright blame. HeeJae would know better, though I think it's very possible we just didn't know this would interact with pth files in any way.

pth files to locations in the workspace root should be fine, since that's how editable installs work (well, ignoring that new PEP).

[debonte]

pth files to locations in the workspace root should be fine, since that's how editable installs work (well, ignoring that new PEP).

Oh, I was thinking that most editable install scenarios would be pointing to files in some other repo/enlistment, and that was why the pth was needed -- so we could actually find those files.

When an editable install's source files are within the workspace, does the presence of the pth enable us to generate/understand correct import paths? Package-relative paths instead of directory-relative paths?

[jakebailey]

It can go both ways; I've seen some projects do this, especially those with a complicated build.

Theoretically it should work, because pth files just tack paths onto sys.path, so are just another import root. But, I would think that if you have a package installed via an editable install that you wouldn't want any heuristic for that package at that point, as the editable install is simulating a real install and the paths better be correct at that point...

In any case, I haven't put myself in the headspace to read over this whole thing and remember what's going on, so forgive me if I'm adding confusion. It's been a while!

[debonte]

If I use explicit relative import, Pylance works normally inside folder2/subfolder but python will not work with explicit relative import in the running scripts or notebooks.

@elashrry, sorry for the long delay on this. I started looking at this again today and my impression both from Pylance's behavior and runtime behavior is that using a relative import (from .module import func) in <workspace-root>\folder2\subfolder\test.py would be correct.

Can you provide more details on the problems that you see at runtime when using an explicit relative import?

[elashrry]

Hello @debonte, thank you for coming back on it.

Explicit relative import as you mentioned would work with Pylance, but if you are running test.py as a script (not as a module), Python will give the error

ImportError: attempted relative import with no known parent package

[debonte]

Ok. Here are two approaches that may solve this for you:

1. Keep the absolute import (from module import func) and create a pyrightconfig.json file that specifies test\folder2\subfolder as an execution environment root directory. There's documentation on this here and some background details here.

{
  "executionEnvironments": [
    {
      "root": "test/folder2/subfolder"
    }
  ]
}

3. Use a relative import (from .module import func) and run test.py via python -m. For example, from the test directory run python -m folder2.subfolder.test.

[elashrry]

Thank you @debonte for the workaround. I want to add that adding

    "python.analysis.extraPaths": [
        "folder2/sub_folder" 
    ]

to the .vscode/settings.json file will work too.