`pipenv lock` nondeterminism with environment markers (again)

[jfly]

I reported this bug about pipenv behaving non-deterministically a while ago: https://github.com/pypa/pipenv/issues/4967. @matteius and I both thought that it was fixed by upgrading to a vendored pip 22.x, but I just checked, and it looks like the problem is back. Actually, it looks like maybe it never went away!

The setup is the same as before: add gevent and sqlalchemy to a Pipfile, and observe that repeated pipenv locks result in the different lockfiles.

Proof/demo

pipenv 2022.8.5

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.8.5 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="platform_python_implementation == 'CPython'"
attempt 2: greenlet markers="platform_python_implementation == 'CPython'"
attempt 3: greenlet markers="platform_python_implementation == 'CPython'"
attempt 4: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"

pipenv 2022.7.24

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.7.24 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="platform_python_implementation == 'CPython'"
attempt 2: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 3: greenlet markers="platform_python_implementation == 'CPython'"

pipenv 2022.4.21

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.4.21 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="python_version >= '3' and (platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32'))))))"
attempt 2: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 3: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 4: greenlet markers="platform_python_implementation == 'CPython'"

pipenv 2022.1.8

The version of pipenv that this was originally reported for (https://github.com/pypa/pipenv/issues/4967)

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.1.8 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="python_version >= '3' and (platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32'))))))"
attempt 2: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 3: greenlet markers="platform_python_implementation == 'CPython'"

[matteius]

Definitely still an issue -- I tired a few things tonight without avail. I wish this code were not so complicated.

[bakhtiary]

~~The point of divergence is when pipenv calls resolver.py in a separate process. The requirements are passed via the PIPENV_PACKAGES env var. Depending on the order in which the requirements are passed, different outcomes are observed. We can replicate this with the following oneliner:~~

 diff <(PIPENV_PACKAGES='sqlalchemy -i https://pypi.org/simple\ngevent -i https://pypi.org/simple' python -m pipenv.resolver) <(PIPENV_PACKAGES='gevent -i https://pypi.org/simple\nsqlalchemy -i https://pypi.org/simple' python -m pipenv.resolver)

~~The quick fix is to make sure the order of the dependencies are the same.~~

~~Secondly, why are the markers different any way?~~

~~Thirdly, should the order of dependencies affect the outcome. This is a more philosophical question when we are working with np-complete problems. Should we sort the dependencies so that we get a particular result every time even if the original order of the dependencies are changed?~~

Edit: running the diff between the same ordered inputs gives the same output so my assumption was wrong:

diff <(PIPENV_PACKAGES='sqlalchemy -i https://pypi.org/simple\ngevent -i https://pypi.org/simple' python -m pipenv.resolver) <(PIPENV_PACKAGES='sqlalchemy -i https://pypi.org/simple\ngevent -i https://pypi.org/simple' python -m pipenv.resolver)

Note that you might have to run the command several times before seeing effects.

[bakhtiary]

gets fixed by: https://github.com/pypa/pipenv/pull/5279

[bakhtiary]

the #5279 PR was accidentally closed. The new PR that fixes this is: https://github.com/pypa/pipenv/pull/5286

[matteius]

I think we also need to sort the constraints. @dqkqd care to open a PR?