pip
pip copied to clipboard
pypa
pip 24.2 networking doesn't work with GraalPy
Description
With truststore being the default in pip 24.2, pip fails to establish a secure connection when using GraalPy.
This is most likely an issue with truststore and/or GraalPy. Since I saw it with a pip update from 24.1.2, opening an issue here.
cc @timfel (GraalPy) cc @sethmlarson (truststore)
Expected behavior
Default settings work without workaround.
pip version
24.2
Python version
graalpy 24.0.2 (python 3.10)
OS
linux
How to Reproduce
https://github.com/mayeut/sandbox/blob/graalpy-pip-24.2/.github/workflows/sandbox.yml
name: Sandbox
on:
push:
workflow_dispatch:
# allow manual runs on branches without a PR
jobs:
sandbox:
runs-on: ubuntu-22.04
steps:
- uses: actions/setup-python@v5
with:
python-version: "graalpy-24.0.2"
- name: test
run: |
python -m pip install -U 'pip==24.2'
python -m pip install -U certifi
Output
https://github.com/mayeut/sandbox/actions/runs/10226713699/job/28297302855
2024-08-03T08:39:43.0604590Z Collecting pip==24.2
2024-08-03T08:39:43.1309965Z Downloading pip-24.2-py3-none-any.whl (1.8 MB)
2024-08-03T08:39:43.4801663Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 6.9 MB/s eta 0:00:00
2024-08-03T08:39:44.0227585Z Installing collected packages: pip
2024-08-03T08:39:44.0238081Z Attempting uninstall: pip
2024-08-03T08:39:44.0314874Z Found existing installation: pip 23.0.1
2024-08-03T08:39:45.2725967Z Uninstalling pip-23.0.1:
2024-08-03T08:39:45.3347495Z Successfully uninstalled pip-23.0.1
2024-08-03T08:39:46.0178591Z Looking for GraalPy patches for pip
2024-08-03T08:39:46.0195671Z We have patches to make this package work on GraalVM for some version(s).
2024-08-03T08:39:46.0197493Z If installing or running fails, consider using one of the versions that we have patches for:
2024-08-03T08:39:46.0214137Z pip == 23.0.1
2024-08-03T08:39:46.0215007Z pip == 22.2.2
2024-08-03T08:39:49.9340828Z Successfully installed pip-24.2
2024-08-03T08:39:50.2013846Z <frozen graalpy.pip_hook>:48: RuntimeWarning: You are using an untested version of pip. GraalPy provides patches and workarounds for a number of packages when used with compatible pip versions. We recommend to stick with the pip version that ships with this version of GraalPy.
2024-08-03T08:39:53.1475503Z WARNING: There was an error checking the latest version of pip.
2024-08-03T08:39:53.2043222Z ERROR: Exception:
2024-08-03T08:39:53.2044059Z Traceback (most recent call last):
2024-08-03T08:39:53.2128945Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
2024-08-03T08:39:53.2130592Z status = _inner_run()
2024-08-03T08:39:53.2132607Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
2024-08-03T08:39:53.2134177Z return self.run(options, args)
2024-08-03T08:39:53.2136003Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
2024-08-03T08:39:53.2141950Z return func(self, options, args)
2024-08-03T08:39:53.2143836Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 379, in run
2024-08-03T08:39:53.2145457Z requirement_set = resolver.resolve(
2024-08-03T08:39:53.2147624Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
2024-08-03T08:39:53.2149284Z result = self._result = resolver.resolve(
2024-08-03T08:39:53.2151116Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
2024-08-03T08:39:53.2152820Z state = resolution.resolve(requirements, max_rounds=max_rounds)
2024-08-03T08:39:53.2154784Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
2024-08-03T08:39:53.2156372Z self._add_to_criteria(self.state.criteria, r, parent=None)
2024-08-03T08:39:53.2158739Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
2024-08-03T08:39:53.2160434Z if not criterion.candidates:
2024-08-03T08:39:53.2162124Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
2024-08-03T08:39:53.2163528Z return bool(self._sequence)
2024-08-03T08:39:53.2165477Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 174, in __bool__
2024-08-03T08:39:53.2167123Z return any(self)
2024-08-03T08:39:53.2168879Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 162, in <genexpr>
2024-08-03T08:39:53.2170771Z return (c for c in iterator if id(c) not in self._incompatible_ids)
2024-08-03T08:39:53.2173390Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 49, in _iter_built
2024-08-03T08:39:53.2175017Z for version, func in infos:
2024-08-03T08:39:53.2177094Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 301, in iter_index_candidate_infos
2024-08-03T08:39:53.2178833Z result = self._finder.find_best_candidate(
2024-08-03T08:39:53.2180652Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 883, in find_best_candidate
2024-08-03T08:39:53.2182445Z candidates = self.find_all_candidates(project_name)
2024-08-03T08:39:53.2184357Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 824, in find_all_candidates
2024-08-03T08:39:53.2186005Z page_candidates = list(page_candidates_it)
2024-08-03T08:39:53.2188112Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/index/sources.py", line 194, in page_candidates
2024-08-03T08:39:53.2189687Z yield from self._candidates_from_page(self._link)
2024-08-03T08:39:53.2191624Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 784, in process_project_url
2024-08-03T08:39:53.2193519Z index_response = self._link_collector.fetch_response(project_url)
2024-08-03T08:39:53.2195462Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 448, in fetch_response
2024-08-03T08:39:53.2197238Z return _get_index_content(location, session=self.session)
2024-08-03T08:39:53.2199115Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 352, in _get_index_content
2024-08-03T08:39:53.2200757Z resp = _get_simple_response(url, session=session)
2024-08-03T08:39:53.2202755Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 131, in _get_simple_response
2024-08-03T08:39:53.2204312Z resp = session.get(
2024-08-03T08:39:53.2205917Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/requests/sessions.py", line 602, in get
2024-08-03T08:39:53.2207378Z return self.request("GET", url, **kwargs)
2024-08-03T08:39:53.2209031Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_internal/network/session.py", line 522, in request
2024-08-03T08:39:53.2210707Z return super().request(method, url, *args, **kwargs)
2024-08-03T08:39:53.2212469Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/requests/sessions.py", line 589, in request
2024-08-03T08:39:53.2213886Z resp = self.send(prep, **send_kwargs)
2024-08-03T08:39:53.2217681Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/requests/sessions.py", line 703, in send
2024-08-03T08:39:53.2219453Z r = adapter.send(request, **kwargs)
2024-08-03T08:39:53.2221279Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/cachecontrol/adapter.py", line 76, in send
2024-08-03T08:39:53.2222977Z resp = super().send(request, stream, timeout, verify, cert, proxies)
2024-08-03T08:39:53.2224903Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/requests/adapters.py", line 667, in send
2024-08-03T08:39:53.2226644Z resp = conn.urlopen(
2024-08-03T08:39:53.2228357Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/urllib3/connectionpool.py", line 715, in urlopen
2024-08-03T08:39:53.2229973Z httplib_response = self._make_request(
2024-08-03T08:39:53.2231059Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/urllib3/connectionpool.py", line 404, in _make_request
2024-08-03T08:39:53.2232191Z self._validate_conn(conn)
2024-08-03T08:39:53.2233306Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/urllib3/connectionpool.py", line 1058, in _validate_conn
2024-08-03T08:39:53.2234231Z conn.connect()
2024-08-03T08:39:53.2235113Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/urllib3/connection.py", line 419, in connect
2024-08-03T08:39:53.2236048Z self.sock = ssl_wrap_socket(
2024-08-03T08:39:53.2237053Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
2024-08-03T08:39:53.2237931Z ssl_sock = _ssl_wrap_socket_impl(
2024-08-03T08:39:53.2238998Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
2024-08-03T08:39:53.2240059Z return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
2024-08-03T08:39:53.2241158Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/truststore/_api.py", line 105, in wrap_socket
2024-08-03T08:39:53.2242186Z _verify_peercerts(ssl_sock, server_hostname=server_hostname)
2024-08-03T08:39:53.2243276Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/truststore/_api.py", line 310, in _verify_peercerts
2024-08-03T08:39:53.2244158Z cert_bytes = _get_unverified_chain_bytes(sslobj)
2024-08-03T08:39:53.2245330Z File "/opt/hostedtoolcache/GraalPy/24.0.2/x64/lib/python3.10/site-packages/pip/_vendor/truststore/_api.py", line 292, in _get_unverified_chain_bytes
2024-08-03T08:39:53.2246504Z unverified_chain = sslobj.get_unverified_chain() or () # type: ignore[attr-defined]
2024-08-03T08:39:53.2247337Z AttributeError: '_SSLSocket' object has no attribute 'get_unverified_chain'
Code of Conduct
- [X] I agree to follow the PSF Code of Conduct.
It looks like get_unverified_chain wasn't a documented (public) API before Python 3.13. So for Python 3.12 and earlier truststore should be prepared for it to not exist.
The code around blob/main/src/pip/_vendor/truststore/_api.py#L277 appears to already be doing version-specific checks, so this should be possible to address.
When GraalPython supports Python 3.13, they will have to add get_unverified_chain to conform to the documented API, that's probably something that should be brought up with them (it's not mentioned in the 3.13 "What's new" as an API change).
Yes, we'll have to add that method in GraalPy. GraalPy's SSL module is based on the JDK's SSL implementation, not OpenSSL directly, but that's for us to worry about. I've opened https://github.com/oracle/graalpython/issues/414 to track this on our side
@sethmlarson SSLSocket.get_unverified_chain is only part of the documented API in Python 3.13+. It seems like this was known when the relevant logic was written, but deemed to be fine as it was implemented (but undocumented) starting with CPython 3.10. Are these APIs needed for truststore to function properly? If so, truststore is going to have to be disabled on GraalPy until Python 3.13+.
same issue that pip cannot work. what should I do to use pip
@ppalantir Downgrade to a version of pip that doesn't enable truststore by default. If you've installed 24.2 already, you should pass --use-deprecated=legacy-certs to get pip temporarily working in order to downgrade to a pip that works for GraalPy.
@ichard26
Are these APIs needed for truststore to function properly?
Indeed, they are required. I can submit a PR that doesn't use Truststore if these APIs aren't detected in the Python runtime in use if that's the approach pip would like to take?
It would be nice if this check could be done on the truststore side so pip doesn't even have to concern itself with platform-specific system SSL differences (which is how pip's integration functions right now), but given the relevant APIs are methods, implementing the checks on pip's end is fine by me if doing it in truststore is too complex. Other maintainers may disagree.
@ichard26 I can certainly add this to Truststore. Let me do that quickly and then pip can rebundle.