pipenv --keep-outdated while switching dependency git location results in conflicting Pipfile and lock

Issue description

Switching a dependency from a fork (eg https://github.com/nicholasserra/twine) to the source (eg https://github.com/pypa/twine) with the --keep-outdated flag results in conflicting Pipfile and Pipfile.lock

Expected result

Expected both the Pipfile and Pipfile.lock to reference https://github.com/pypa/twine

Actual result

Pipfile references the new source given (https://github.com/pypa/twine) but the Pipfile.lock still references the old fork url and commit (https://github.com/nicholasserra/twine).

Steps to replicate

Starting with a blank project, no Pipfile or lock.
Install a github dependency at a commit from a fork

pipenv install -e git+https://github.com/nicholasserra/twine.git@668f4db44d2965d9009308d96e13776069d050db#egg=twine

Pipfile shows the following:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
twine = {editable = true, ref = "668f4db44d2965d9009308d96e13776069d050db", git = "https://github.com/nicholasserra/twine.git"}

Pipfile.lock shows the following (clutter removed)

{
    "_meta": {
        "hash": {
            "sha256": "027966d540aa935f91d54fd1141002805ec638da730d755661de05b233ca879e"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
    ...
        "twine": {
            "editable": true,
            "git": "https://github.com/nicholasserra/twine.git",
            "ref": "668f4db44d2965d9009308d96e13776069d050db"
        },
    ...
    },
    "develop": {}
}

Update the dependency to reference the source github url at a commit:

pipenv install -e git+https://github.com/pypa/twine.git@8f5e5d6d42d582ef3ea6ef07da277e0cabd22fd2#egg=twine --keep-outdated

The Pipfile correctly gets updated:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
twine = {editable = true, ref = "8f5e5d6d42d582ef3ea6ef07da277e0cabd22fd2", git = "https://github.com/pypa/twine.git"}

The Pipfile.lock still references the old url and commit:

{
    "_meta": {
        "hash": {
            "sha256": "dc1ca95272a1bbd8a7fe0c22e1f5e9a7b4fab405c67d32b96a3f9534b833fbaf"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
    ...
        "twine": {
            "editable": true,
            "git": "https://github.com/nicholasserra/twine.git",
            "ref": "668f4db44d2965d9009308d96e13776069d050db"
        },
    ...
    },
    "develop": {}
}

$ pipenv --support

Pipenv version: '2022.6.7'

Pipenv location: '/Users/nick/Code/pipenv-bug/venv/lib/python3.8/site-packages/pipenv'

Python location: '/Users/nick/Code/pipenv-bug/venv/bin/python'

Python installations found:

3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python3
3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python
3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python3
3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python
3.8.2: /Users/nick/.pyenv/versions/3.8.2/bin/python3
3.7.3: /usr/bin/python3
3.7.0: /Users/nick/.pyenv/versions/3.7.0/bin/python3
3.6.1: /Users/nick/.pyenv/versions/3.6.1/bin/python3
3.4.3: /usr/local/bin/python3
3.4.3: /usr/local/bin/python3.4m
3.4.3: /usr/local/bin/python3.4
2.7.16: /usr/bin/python
2.7.16: /usr/bin/python2
2.7.16: /usr/bin/python2.7
2.7.13: /usr/local/bin/python
2.7.13: /usr/local/bin/python2
2.7.13: /usr/local/bin/python2.7
2.7.13: /Users/nick/.pyenv/versions/2.7.13/bin/python
2.7.10: /Users/nick/.pyenv/versions/2.7.10/bin/python

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.2',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '19.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 19.6.0: Mon Apr 18 21:50:40 PDT '
                     '2022; root:xnu-6153.141.62~1/RELEASE_X86_64',
 'python_full_version': '3.8.2',
 'python_version': '3.8',
 'sys_platform': 'darwin'}

System environment variables:

MANPATH
rvm_bin_path
TERM_PROGRAM
NVM_CD_FLAGS
GEM_HOME
TERM
SHELL
IRBRC
TMPDIR
NVM_PATH
TERM_PROGRAM_VERSION
OLDPWD
MY_RUBY_HOME
TERM_SESSION_ID
NVM_DIR
USER
_system_type
rvm_path
SSH_AUTH_SOCK
__CF_USER_TEXT_ENCODING
VIRTUAL_ENV
rvm_prefix
PATH
NVM_NODEJS_ORG_MIRROR
PWD
LANG
ITERM_PROFILE
_system_arch
XPC_FLAGS
PS1
PYTHONDONTWRITEBYTECODE
_system_version
XPC_SERVICE_NAME
rvm_version
PYENV_SHELL
SHLVL
HOME
COLORFGBG
LC_TERMINAL_VERSION
ITERM_SESSION_ID
LOGNAME
GEM_PATH
GOPATH
NVM_BIN
NVM_IOJS_ORG_MIRROR
LC_TERMINAL
RUBY_VERSION
_system_name
COLORTERM
_
PIP_SHIMS_BASE_MODULE
PIP_DISABLE_PIP_VERSION_CHECK
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /Users/nick/Code/pipenv-bug/venv/bin:/Users/nick/.pyenv/shims:/Users/nick/.nvm/versions/node/v10.15.1/bin:/Applications/Postgres.app/Contents/Versions/9.4/bin:/Users/nick/.rvm/gems/ruby-2.3.0/bin:/Users/nick/.rvm/gems/ruby-2.3.0@global/bin:/Users/nick/.rvm/rubies/ruby-2.3.0/bin:/usr/local/heroku/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:/Applications/Wireshark.app/Contents/MacOS:/usr/local/git/bin:/Users/nick/.rvm/bin:/Users/nick/.rvm/bin:/Users/nick/bin
SHELL: /bin/bash
LANG: en_US.UTF-8
PWD: /Users/nick/Code/pipenv-bug
VIRTUAL_ENV: /Users/nick/Code/pipenv-bug/venv

Contents of Pipfile ('/Users/nick/Code/pipenv-bug/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
twine = {editable = true, ref = "8f5e5d6d42d582ef3ea6ef07da277e0cabd22fd2", git = "https://github.com/pypa/twine.git"}

[dev-packages]

[requires]
python_version = "3.8"

Contents of Pipfile.lock ('/Users/nick/Code/pipenv-bug/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "dc1ca95272a1bbd8a7fe0c22e1f5e9a7b4fab405c67d32b96a3f9534b833fbaf"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "bleach": {
            "hashes": [
                "sha256:085f7f33c15bd408dd9b17a4ad77c577db66d76203e5984b1bd59baeee948b2a",
                "sha256:0d03255c47eb9bd2f26aa9bb7f2107732e7e8fe195ca2f64709fcf3b0a4a085c"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==5.0.1"
        },
        "certifi": {
            "hashes": [
                "sha256:84c85a9078b11105f04f3036a9482ae10e4621616db313fe045dd24743a0820d",
                "sha256:fe86415d55e84719d75f8b69414f6438ac3547d2078ab91b67e779ef69378412"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==2022.6.15"
        },
        "charset-normalizer": {
            "hashes": [
                "sha256:2857e29ff0d34db842cd7ca3230549d1a697f96ee6d3fb071cfa6c7393832597",
                "sha256:6881edbebdb17b39b4eaaa821b438bf6eddffb4468cf344f09f89def34a8b1df"
            ],
            "markers": "python_full_version >= '3.5.0'",
            "version": "==2.0.12"
        },
        "colorama": {
            "hashes": [
                "sha256:854bf444933e37f5824ae7bfc1e98d5bce2ebe4160d46b5edf346a89358e99da",
                "sha256:e6c6b4334fc50988a639d9b98aa429a0b57da6e17b9a44f0451f930b6967b7a4"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==0.4.5"
        },
        "commonmark": {
            "hashes": [
                "sha256:452f9dc859be7f06631ddcb328b6919c67984aca654e5fefb3914d54691aed60",
                "sha256:da2f38c92590f83de410ba1a3cbceafbc74fee9def35f9251ba9a971d6d66fd9"
            ],
            "version": "==0.9.1"
        },
        "docutils": {
            "hashes": [
                "sha256:23010f129180089fbcd3bc08cfefccb3b890b0050e1ca00c867036e9d161b98c",
                "sha256:679987caf361a7539d76e584cbeddc311e3aee937877c87346f31debc63e9d06"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==0.18.1"
        },
        "idna": {
            "hashes": [
                "sha256:84d9dd047ffa80596e0f246e2eab0b391788b0503584e8945f2368256d2735ff",
                "sha256:9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d"
            ],
            "markers": "python_full_version >= '3.5.0'",
            "version": "==3.3"
        },
        "importlib-metadata": {
            "hashes": [
                "sha256:637245b8bab2b6502fcbc752cc4b7a6f6243bb02b31c5c26156ad103d3d45670",
                "sha256:7401a975809ea1fdc658c3aa4f78cc2195a0e019c5cbc4c06122884e9ae80c23"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==4.12.0"
        },
        "keyring": {
            "hashes": [
                "sha256:372ff2fc43ab779e3f87911c26e6c7acc8bb440cbd82683e383ca37594cb0617",
                "sha256:3ac00c26e4c93739e19103091a9986a9f79665a78cf15a4df1dba7ea9ac8da2f"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==23.6.0"
        },
        "pkginfo": {
            "hashes": [
                "sha256:848865108ec99d4901b2f7e84058b6e7660aae8ae10164e015a6dcf5b242a594",
                "sha256:a84da4318dd86f870a9447a8c98340aa06216bfc6f2b7bdc4b8766984ae1867c"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
            "version": "==1.8.3"
        },
        "pygments": {
            "hashes": [
                "sha256:5eb116118f9612ff1ee89ac96437bb6b49e8f04d8a13b514ba26f620208e26eb",
                "sha256:dc9c10fb40944260f6ed4c688ece0cd2048414940f1cea51b8b226318411c519"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==2.12.0"
        },
        "readme-renderer": {
            "hashes": [
                "sha256:73b84905d091c31f36e50b4ae05ae2acead661f6a09a9abb4df7d2ddcdb6a698",
                "sha256:a727999acfc222fc21d82a12ed48c957c4989785e5865807c65a487d21677497"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==35.0"
        },
        "requests": {
            "hashes": [
                "sha256:bc7861137fbce630f17b03d3ad02ad0bf978c844f3536d0edda6499dafce2b6f",
                "sha256:d568723a7ebd25875d8d1eaf5dfa068cd2fc8194b2e483d7b1f7c81918dbec6b"
            ],
            "markers": "python_version >= '3.7' and python_version < '4'",
            "version": "==2.28.0"
        },
        "requests-toolbelt": {
            "hashes": [
                "sha256:380606e1d10dc85c3bd47bf5a6095f815ec007be7a8b69c878507068df059e6f",
                "sha256:968089d4584ad4ad7c171454f0a5c6dac23971e9472521ea3b6d49d610aa6fc0"
            ],
            "version": "==0.9.1"
        },
        "rfc3986": {
            "hashes": [
                "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd",
                "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==2.0.0"
        },
        "rich": {
            "hashes": [
                "sha256:4c586de507202505346f3e32d1363eb9ed6932f0c2f63184dea88983ff4971e2",
                "sha256:d2bbd99c320a2532ac71ff6a3164867884357da3e3301f0240090c5d2fdac7ec"
            ],
            "markers": "python_version < '4' and python_full_version >= '3.6.3'",
            "version": "==12.4.4"
        },
        "setuptools": {
            "hashes": [
                "sha256:990a4f7861b31532871ab72331e755b5f14efbe52d336ea7f6118144dd478741",
                "sha256:c1848f654aea2e3526d17fc3ce6aeaa5e7e24e66e645b5be2171f3f6b4e5a178"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==62.6.0"
        },
        "six": {
            "hashes": [
                "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926",
                "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.16.0"
        },
        "tqdm": {
            "hashes": [
                "sha256:40be55d30e200777a307a7585aee69e4eabb46b4ec6a4b4a5f2d9f11e7d5408d",
                "sha256:74a2cdefe14d11442cedf3ba4e21a3b84ff9a2dbdc6cfae2c34addb2a14a5ea6"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==4.64.0"
        },
        "twine": {
            "editable": true,
            "git": "https://github.com/nicholasserra/twine.git",
            "ref": "668f4db44d2965d9009308d96e13776069d050db"
        },
        "typing-extensions": {
            "hashes": [
                "sha256:6657594ee297170d19f67d55c05852a874e7eb634f4f753dbd667855e07c1708",
                "sha256:f1c24655a0da0d1b67f07e17a5e6b2a105894e6824b92096378bb3668ef02376"
            ],
            "markers": "python_version < '3.9'",
            "version": "==4.2.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:44ece4d53fb1706f667c9bd1c648f5469a2ec925fcf3a776667042d645472c14",
                "sha256:aabaf16477806a5e1dd19aa41f8c2b7950dd3c746362d7e3223dbe6de6ac448e"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
            "version": "==1.26.9"
        },
        "webencodings": {
            "hashes": [
                "sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78",
                "sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923"
            ],
            "version": "==0.5.1"
        },
        "zipp": {
            "hashes": [
                "sha256:56bf8aadb83c24db6c4b577e13de374ccfb67da2078beba1d037c17980bf43ad",
                "sha256:c4f6e5bbf48e74f7a38e7cc5b0480ff42b0ae5178957d564d18932525d5cf099"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==3.8.0"
        }
    },
    "develop": {}
}

Jun 27 '22 21:06 nicholasserra

I am guessing that it has something to do with the version number not changing in the fork, so it keeps the outdated requirement. To be honest, I find the --keep-outdated implementation to be very much a hack that can lead to unexpected requirement update behaviors. I recommend not using it if possible and allowing your requirements to update to the specifiers you have outlined in your Pipfile by simply using pipenv lock or pipenv install without the --keep-outdated flag.

Jun 27 '22 22:06 matteius

@nicholasserra --keep-outdated and likely --selective-upgrade are deprecated/slated for removal. I've been working on an alternative command that solves the kind of problems these other flags caused. Checkout out: https://github.com/pypa/pipenv/pull/5617

Any feedback appreicated!

Feb 20 '23 00:02 matteius

pipenv pipenv copied to clipboard

--keep-outdated while switching dependency git location results in conflicting Pipfile and lock

Issue description

Expected result

Actual result

Steps to replicate

pipenv
pipenv copied to clipboard