PEP660 editable VCS dependencies not reinstalled correctly

[edmorley]

Issue description

For editable VCS (eg Git) dependencies, re-running pipenv install does not reinstall the dependency if the linked source repository checkout is missing (eg if it has been moved to a different path, or otherwise removed).

This appears to only affect PEP 660 style editable installs.

Expected result

For pipenv install to reinstall an editable VCS dependency, if the src/ directory containing the repository checkout has been removed since the last time pipenv install was run.

For example, this works with pip:

FROM python:3.13
WORKDIR /testcase
RUN pip install -e git+https://github.com/benoitc/[email protected]#egg=gunicorn
RUN python -c 'import gunicorn'
RUN rm -rf src/
RUN pip install -e git+https://github.com/benoitc/[email protected]#egg=gunicorn
RUN python -c 'import gunicorn'

Actual result

Pipenv doesn't reinstall the editable VCS dependency - the pipenv install is a no-op:

#14 [ 9/10] RUN pipenv install --system --verbose
#14 0.282 Installing dependencies from Pipfile.lock (13dee2)...
#14 DONE 0.3s

Which then results in a ModuleNotFoundError:

#15 [10/10] RUN python -c 'import gunicorn'
#15 0.130 Traceback (most recent call last):
#15 0.132   File "<string>", line 1, in <module>
#15 0.132     import gunicorn
#15 0.132 ModuleNotFoundError: No module named 'gunicorn'

Steps to replicate

Create a Dockerfile, with contents:

FROM python:3.13
WORKDIR /testcase

COPY <<EOF Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
gunicorn = {git = "git+https://github.com/benoitc/gunicorn", ref = "23.0.0", editable = true}
EOF

COPY <<EOF Pipfile.lock
{
    "_meta": {
        "hash": {
            "sha256": "0979fd8702b00f90e3e2e93d14919cf23f76b992c80662d6e610dc850d13dee2"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "gunicorn": {
            "editable": true,
            "git": "git+https://github.com/benoitc/gunicorn",
            "markers": "python_version >= '3.7'",
            "ref": "411986d6191114dd1d1bbb9c72c948dbf0ef0425"
        },
        "packaging": {
            "hashes": [
                "sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759",
                "sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f"
            ],
            "markers": "python_version >= '3.8'",
            "version": "==24.2"
        }
    },
    "develop": {}
}
EOF

RUN pip install pipenv==v2024.4.1
RUN pipenv install --system --verbose

# This works
RUN python -c 'import gunicorn'

# Remove the gunicorn source checkout
RUN rm -rf src/

# Reinstall
RUN pipenv install --system --verbose

# The src dir is missing here since pipenv didn't reinstall the editable VCS dependency
RUN ls -al

# And so this fails with a `ModuleNotFoundError`
RUN python -c 'import gunicorn'

Then run it with: docker build . --progress plain --no-cache

Additional findings

If I downgrade the gunicorn version, to one that uses setup.py instead of pyproject.toml (by changing ref = "23.0.0" to ref = "21.2.0") and set ENV PIP_USE_PEP517=0 to force a legacy setup.py develop editable install then this issue no longer reproduces.

With the legacy editable install, site-packages contains:

easy-install.pth
gunicorn.egg-link

Whereas when using the PEP517 / PEP660 install (as per the Dockerfile above), site-packages instead contains:

gunicorn-23.0.0.dist-info/
__editable__.gunicorn-23.0.0.pth
__editable___gunicorn_23_0_0_finder.py

As such, I've presuming this might be an issue with how Pipenv handles editable VCS dependencies specifically when they use the new PEP660 editable interface?

Is the issue somewhere here? https://github.com/pypa/pipenv/blob/46bc4ef3b482f1a4fbffee14056780c9c9a8133a/pipenv/environment.py#L730-L761

[matteius]

@edmorley Could you check the open PR I added -- I think conceptually it makes sense; we'll see if the new test case passes too.