pypa
Using a private package index with pip 24.2 results in `ValueError: check_hostname requires server_hostname`
Description
Attempting to install a package from a private package index with pip 24.2 results in a ValueError: check_hostname requires server_hostname error.
I am using the command below:
pip install <package name> --index-url=<private package index>
Expected behavior
Package is downloaded from the index without an error occuring.
pip version
24.2
Python version
3.12
OS
Linux
How to Reproduce
- Install pip:
pip install pip==24.2 - Try to install a package from the private index. For example with
pip install pip==24.1 --index-url=https://163.219.218.169/root/pypi --no-cache-dir - The
ValueError: check_hostname requires server_hostnameexception occurs.
Output
Full traceback
WARNING: There was an error checking the latest version of pip.
ERROR: Exception:
Traceback (most recent call last):
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
status = _inner_run()
^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
return self.run(options, args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 379, in run
requirement_set = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
return bool(self._sequence)
^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 174, in __bool__
return any(self)
^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 162, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 49, in _iter_built
for version, func in infos:
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 301, in iter_index_candidate_infos
result = self._finder.find_best_candidate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/index/package_finder.py", line 883, in find_best_candidate
candidates = self.find_all_candidates(project_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/index/package_finder.py", line 824, in find_all_candidates
page_candidates = list(page_candidates_it)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/index/sources.py", line 194, in page_candidates
yield from self._candidates_from_page(self._link)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/index/package_finder.py", line 784, in process_project_url
index_response = self._link_collector.fetch_response(project_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/index/collector.py", line 448, in fetch_response
return _get_index_content(location, session=self.session)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/index/collector.py", line 352, in _get_index_content
resp = _get_simple_response(url, session=session)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/index/collector.py", line 131, in _get_simple_response
resp = session.get(
^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/network/session.py", line 522, in request
return super().request(method, url, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/urllib3/connectionpool.py", line 715, in urlopen
httplib_response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/urllib3/connectionpool.py", line 404, in _make_request
self._validate_conn(conn)
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/urllib3/connectionpool.py", line 1058, in _validate_conn
conn.connect()
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/urllib3/connection.py", line 419, in connect
self.sock = ssl_wrap_socket(
^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 453, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 495, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/truststore/_api.py", line 96, in wrap_socket
ssl_sock = self._ctx.wrap_socket(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/.pyenv/versions/3.12.3/lib/python3.12/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hoel/.pyenv/versions/3.12.3/lib/python3.12/ssl.py", line 970, in _create
raise ValueError("check_hostname requires server_hostname")
ValueError: check_hostname requires server_hostname
Other
I am still in the process of setting up the https part of my private package index, and while I was able to use it to some extend with pip==24.1, I had to manually point to the CA certificate. So I cannot exclude the possibility of a wrong setup on my part.
$ pip install pip==24.2 --index-url=https://163.219.218.169/root/pypi --no-cache-dir --use-feature=truststore
Could not Could not fetch URL https://163.219.218.169/root/pypi/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='163.219.218.169', port=443): Max retries exceeded with url: /root/pypi/pip/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)'))) - skipping
$ pip install pip==24.2 --index-url=https://163.219.218.169/root/pypi --no-cache-dir --cert assets/ca.pem
Successfully installed pip-24.2
Code of Conduct
- [x] I agree to follow the PSF Code of Conduct.
Could you try passing --use-deprecated=legacy-certs to the pip invocation? If that works, then we can narrow down the issue to the truststore feature (which seems likely given that pip 24.1 works which has truststore disabled by default). cc @sethmlarson
I wasn't able to get the ValueError you're running into to reproduce, is there any other flag you're passing to pip or environment variable that would configure/disable certificate verification?
I tried using TestPyPI as an example "index-url", maybe it only triggers for other indices?
$ python -m pip --version
pip 24.2 from /tmp/pip-12906/venv/lib/python3.12/site-packages/pip (python 3.12)
$ python -m pip install urllib3 --index-url=https://test.pypi.org/simple --no-cache-dir
(succeeds)
Could you try passing --use-deprecated=legacy-certs to the pip invocation? If that works, then we can narrow down the issue to the truststore feature (which seems likely given that pip 24.1 works which has truststore disabled by default).
If I use --use-deprecated=legacy-certs, then the behavior becomes the same as with pip 24.1. It works if I give the pass to the CA using --cert (I guess that's because I haven't configured things properly, although I don't know what I'm missing).
I wasn't able to get the ValueError you're running into to reproduce, is there any other flag you're passing to pip or environment variable that would configure/disable certificate verification?
Not that I'm aware of. I saw in this issue that it got linked to proxy settings. I am behind a proxy, however I am unsetting all the proxy environment variable when running the pip commands (since the private index is in the local network).
I tried using TestPyPI as an example "index-url", maybe it only triggers for other indices?
I tried your command, and it worked without issue. I'm not sure how relevant it is, but I set the proxy environment variables when doing it. I'll try to replicate the issue in a simplified environment.
(I guess that's because I haven't configured things properly, although I don't know what I'm missing).
Well, before pip 24.2, pip would only verify certificates against its vendored certificate bundle. That's what certifi is: it's a Python repackaging of Mozilla's certificate bundle. For a while, pip gained the ability to integrate with truststore which uses the system certificate store. However, this feature was disabled by default and had to be enabled manually until pip 24.2. In theory, "it should just work" if your private CA is configured properly at the system level with truststore enabled, but evidently somewhere in the chain (:P) it's not working...
A simplified or MRE would be greatly appreciated.
@ichard26 I've been able to reproduce the error using this repo.
I ran it at home on my personal computer, so no proxy was involved.
@sethmlarson while I have your attention, perhaps you could look at this too? :)
There is an attached reproduction repository, but I was able to reproduce the error by simply pointing pip to a 127.0.0.1 HTTPS index with a custom CA/cert. I wasn't able to figure out what exactly goes wrong, but it seems like with truststore enabled, the check_hostname session attribute remains true despite urllib3 attempting to disable it.
Heya, any updates @sethmlarson? It's too late for the 24.3 cycle, but I'd love to make truststore rock solid on supported platforms in pip 25.0.
I haven't dug into this issue recently, but the latest release of truststore did make some changes to check_hostname, might be worth checking again.
I'm seeing this error also with a self-signed root CA and private PyPi registry. Usually setting PIP_CERT works, but not with v24.2 and v24.3. I keep getting a ValueError: check_hostname requires server_hostname error.
File "/server-082/workspace/project-ea52f8/venv/lib/python3.12/site-packages/pip/_vendor/truststore/_api.py", line 96, in wrap_socket
ssl_sock = self._ctx.wrap_socket(
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/tools/Python/3.12.7/x64/lib/python3.12/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/tools/Python/3.12.7/x64/lib/python3.12/ssl.py", line 969, in _create
raise ValueError("check_hostname requires server_hostname")
ValueError: check_hostname requires server_hostname
