pypa
Allow pipx to use sytem certificates
Describe the bug
pipx install fails due to corporate network/proxy with self-signed certificate.
From the log:
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
ERROR: Could not find a version that satisfies the requirement hatch (from versions: none)
ERROR: No matching distribution found for hatch
Note that if I create a venv, things work as expected. So the system python/pip handles the (installed) self-signed cert:
syncer on chore/setup-hatch [!?]
❯ python3 -m venv .venv
syncer on chore/setup-hatch [!?] took 2s
❯ source .venv/bin/activate
syncer on chore/setup-hatch [!?] via 🐍 v3.10.6 (.venv)
❯ pip install hatch
Collecting hatch
Downloading hatch-1.6.3-py3-none-any.whl (84 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.0/85.0 KB 2.7 MB/s eta 0:00:00
Collecting platformdirs>=2.5.0
Downloading platformdirs-3.2.0-py3-none-any.whl (14 kB)
**snip**
Collecting pycparser
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 14.3 MB/s eta 0:00:00
Using legacy 'setup.py install' for pyperclip, since package 'wheel' is not installed.
Installing collected packages: rfc3986, pyperclip, ptyprocess, distlib, zipp, tomlkit, tomli-w, tomli, sniffio, shellingham, pygments, pycparser, pluggy, platformdirs, pexpect, pathspec, packaging, more-itertools, mdurl, jeepney, idna, h11, filelock, editables, click, certifi, virtualenv, userpath, markdown-it-py, jaraco.classes, importlib-metadata, hyperlink, hatchling, cffi, anyio, rich, httpcore, cryptography, SecretStorage, httpx, keyring, hatch
Running setup.py install for pyperclip ... done
Successfully installed SecretStorage-3.3.3 anyio-3.6.2 certifi-2022.12.7 cffi-1.15.1 click-8.1.3 cryptography-40.0.1 distlib-0.3.6 editables-0.3 filelock-3.10.7 h11-0.14.0 hatch-1.6.3 hatchling-1.13.0 httpcore-0.16.3 httpx-0.23.3 hyperlink-21.0.0 idna-3.4 importlib-metadata-6.1.0 jaraco.classes-3.2.3 jeepney-0.8.0 keyring-23.13.1 markdown-it-py-2.2.0 mdurl-0.1.2 more-itertools-9.1.0 packaging-23.0 pathspec-0.11.1 pexpect-4.8.0 platformdirs-3.2.0 pluggy-1.0.0 ptyprocess-0.7.0 pycparser-2.21 pygments-2.14.0 pyperclip-1.8.2 rfc3986-1.5.0 rich-13.3.3 shellingham-1.5.0.post1 sniffio-1.3.0 tomli-2.0.1 tomli-w-1.0.0 tomlkit-0.11.7 userpath-1.8.0 virtualenv-20.21.0 zipp-3.15.0
syncer on chore/setup-hatch [!?] via 🐍 v3.10.6 (.venv) took 17s
❯
I'm using the zipapp (pipx.pyz), version (1.2.0)
My environment is Ubuntu 22.04 running in WSL on Windows 10.
Python is 3.10.6
How to reproduce
syncer on chore/setup-hatch [!?]
❯ python3 /home/jim/dev/afa/syncer/.deps/pipx/1.2.0/pipx.pyz --verbose install hatch
usage: pipx.pyz [-h] [--version] {install,uninject,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall,reinstall-all,list,run,runpip,ensurepath,environment,completions} ...
pipx.pyz: error: unrecognized arguments: --verbose
To reproduce I guess you have to be in an environment that has a proxy with a self-signed cert.
Expected behavior pipx successfully installs package.
Adding export PIP_CERT="/etc/ssl/certs" seems to have solved it.
However, that seems like a hack.
Happy to help troubleshoot further.
I'm having this issue too. I can get around this for standard pip using the pip-system-certs package.
pip install pip-system-certs config --trusted-host pypi.org --trusted-host files.pythonhosted.org
Unfortunately pipx doesn't seem to respect it. I'd try the above workaround but I'm on MacOS and I can't seem to find where it stores its certs.
This can be solved by setting the global cert flag in pip (where ca_certs is the folder containing your custom cert):
pip config set global.cert ~/ca_certs
or by setting the SSL_CERT_FILE environment variable:
export CERT_PATH=/etc/ssl/certs/ZscalerRootCA.pem
export CERT_DIR=/etc/ssl/certs/
export SSL_CERT_FILE=${CERT_PATH}
export SSL_CERT_DIR=${CERT_DIR}
export REQUESTS_CA_BUNDLE=${CERT_PATH}
Source: zscaler docs
This can be solved by setting the global cert flag in pip (where ca_certs is the folder containing your custom cert):
pip config set global.cert ~/ca_certsor by setting the SSL_CERT_FILE environment variable:
export CERT_PATH=/etc/ssl/certs/ZscalerRootCA.pem export CERT_DIR=/etc/ssl/certs/ export SSL_CERT_FILE=${CERT_PATH} export SSL_CERT_DIR=${CERT_DIR} export REQUESTS_CA_BUNDLE=${CERT_PATH}Source: zscaler docs
While a good solution, isn't it more of a workaround? Shouldn't pipx solve this? At least in my case where pip handled it without any extras.