Install private packages broken in 2023.8.19 version

[jcozar87]

Issue description

I am using a private repository for my python packages in AWS (service AWS CodeArtifact). I created my Pipfile according to the documentation. It works fine from versions 2022.7.4 to 2023.7.23, but from version 2023.8.19 and above it fails (it does not find my private packages).

I checked the release notes of version 2023.8.19, but I don't see any change or incompatibility.

Expected result

To install my private package successfully using pipenv install and to create the Pipfile.lock file using pipenv lock.

Actual result

This is the traceback of the error:

ERROR: No matching distribution found for aws_mypackage==0.1.0

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 224, in install
    categories=state.installstate.categories,
  File "/usr/local/lib/python3.7/site-packages/pipenv/routines/install.py", line 184, in do_install
    categories=categories,
  File "/usr/local/lib/python3.7/site-packages/pipenv/routines/install.py", line 685, in do_init
    categories=categories,
  File "/usr/local/lib/python3.7/site-packages/pipenv/routines/lock.py", line 76, in do_lock
    old_lock_data=old_lock_data,
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils/resolver.py", line 782, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils/resolver.py", line 651, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

Steps to replicate

I am using the official image from dockerhub of python3.7, python3.8, python3.9, python3.10 and python3.11. The error is the same in all versions.

For example, to run the python3.7 container:

docker run -it --rm -e AWS_PIP_INDEX=$AWS_PIP_INDEX -v $(pwd):/ python:3.7 bash

Then install pipenv: python3 -m pip install pipenv==2023.8.19 and run pipenv lock

My Pipfile is:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "$AWS_PIP_INDEX"
verify_ssl = true
name = "aws"

[packages]
aws_mypackage = {version="0.1.0", index="aws"}

The command that returns the error is, for example AWS_PIP_INDEX=... pipenv lock, where you need to set the correct string for the env variable. I also tried to write the string directly in the Pipfile instead of the environment variable, but the error is the same.

---

I run the $ pipenv --support command using python3.7 and pipenv==2023.8.19:

$ pipenv --support

Pipenv version: '2023.8.19'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/bin/python3'

OS Name: 'posix'

User pip version: '23.2.1'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.17',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.10.76-linuxkit',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT Mon Nov 8 11:22:26 UTC 2021',
 'python_full_version': '3.7.17',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

• _
• OLDPWD
• PATH
• PYTHON_GET_PIP_URL
• PYTHON_GET_PIP_SHA256
• PYTHON_PIP_VERSION
• SHLVL
• TERM
• GPG_KEY
• LANG
• HOME
• PYTHON_SETUPTOOLS_VERSION
• PWD
• PYTHON_VERSION
• HOSTNAME
• AWS_PIP_INDEX
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
• LANG: C.UTF-8
• PWD: /app

---

Contents of Pipfile ('/app/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "$AWS_PIP_INDEX"
verify_ssl = true
name = "aws"

[packages]
aws_mypackage = {version="0.1.0", index="aws"}

[dev-packages]

[requires]
python_version = "3.7"

[matteius]

2023.8.19 is not the latest version though.

[jcozar87]

Hi @matteius , thank you for your time!

I tried also with the latest version too (2023.10.3), but the problem is the same. I give you the oldest version which it crashes (2023.8.19), so you would have more information about where the problem is.

[matteius]

Unfortunately I don't have a AWS CodeArtifact to test against, so its rather limiting.

[ring-pete]

Just adding I can confirm this issue on my systems too which have a pretty similar configuration. If it would help to have access to a private PyPi server to help test this I am happy to provision one for debugging and validation purposes.

[matteius]

@ring-pete that would be great -- I don't have an AWS account and would rather not incur the expense of one (I run my projects in Digital Ocean). I did create a GCP account to diagnose a similar issue but I got away with free credits and that issue was resolved. Would be great to get AWS squared away since it sounds like there may be an issue there.

[ring-pete]

Hey there, I should have been more clear - I can reproduce this with our private PyPi repository (in this case its a Sonatype Nexus server). There are no AWS specific bits used in our case. I've created a repository on our server and will privately send you the credentials and an example of how we use it.

[monkeyman192]

FWIW I am able to download files from our private nexus repo with the latest pipenv version on python 3.9:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[[source]]
name = "nexus"
url = "$NEXUS_URL"
verify_ssl = true

[packages]
my_package = {version=">=0.8.0", index="nexus"}

[requires]
python_version = "3.9"

And ran with NEXUS_URL=... pipenv lock

HOWEVER... If I change the versions to be the same as in the OP, I get a failure (ie. my_package = {version="0.8.0", index="nexus"} instead of my_package = {version=">=0.8.0", index="nexus"} The error is a little different, but maybe that's just because of the version difference... So maybe this is just an issue with needing == or similar?

[matteius]

@monkeyman192 yeah I would expect a specifier is required, so either ">=0.8.0 or "==0.8.0 but not 0.8.0

[slackcc]

I am experiencing the same issue. If I revert back to 2023.7.23 everything works fine, if I update to 2023.8.19 or newer it fails. I can confirm that adding a specifier resolves the issue (tested with the latest version, 2023.11.15 at the time I'm writing this). This is an acceptable workaround, but this should still likely be considered a bug since just specifying a version number, without specifier, has always worked in the past and works with standard pip.

[jcozar87]

I tried again using version 2023.12.1 of pipenv and it works with AWS CodeArtifact. Thank you very much for your effort evolving and improving this awesome python package! :-)