requests icon indicating copy to clipboard operation
requests copied to clipboard

self sign certificate validation error (requests-2.23.5, urllib3-2.5.0)

Open vaibhav-chaturvedi opened this issue 2 months ago • 4 comments

Self sign certificate validation is getting failed when using requests 2.23.5 with urllib3 2.5.0. It works with urllib3 1.26.20 version. Any version above 1.26.20 fails to validate certificates.

Expected Result

Certificate should be validated.

Actual Result

Traceback (most recent call last): File "/home/user/.local/lib/python3.9/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( File "/home/user/.local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 843, in urlopen retries = retries.increment( File "/home/user/.local/lib/python3.9/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='', port=): Max retries exceeded with url: /api/v3/health (Caused by SSLError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for ''. (_ssl.c:1129)")))

Reproduction Steps

import requests
requests.get(url, verify=<certificate_path>)


System Information

$ python -m requests.help
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.4.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.11"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.16"
  },
  "platform": {
    "release": "5.15.0-1092-azure",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.32.5"
  },
  "system_ssl": {
    "version": "101010ef"
  },
  "urllib3": {
    "version": "2.5.0"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

vaibhav-chaturvedi avatar Nov 11 '25 14:11 vaibhav-chaturvedi

Hi I am seeing a similar issue!

I work behind a corporate proxy and as such I normally use my own CA bundle.

import requests
import os

os.environ["REQUESTS_CA_BUNDLE"] = "cert.cer"

r = requests.get("https://google.com")

print(r.status_code)

When I run this using Python 3.12.9 with the following versions and their dependencies:

Using Python 3.12.9 environment at: .venv-312
Package            Version
------------------ ----------
certifi            2025.11.12
charset-normalizer 3.4.4
idna               3.11
requests           2.32.5
urllib3            2.5.0

everything works fine:

$ uv run python recreate.py
200

running the same script with Python 3.13.2 with the following versions and their dependencies:

Using Python 3.13.2 environment at: .venv-313
Package            Version
------------------ ----------
certifi            2025.11.12
charset-normalizer 3.4.4
idna               3.11
requests           2.32.5
urllib3            2.5.0

I get the following error traceback:

Traceback (most recent call last):
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\connectionpool.py", line 464, in _make_request
    self._validate_conn(conn)
    ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\connectionpool.py", line 1093, in _validate_conn
    conn.connect()
    ~~~~~~~~~~~~^^
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\connection.py", line 790, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
        sock=sock,
    ...<14 lines>...
        assert_fingerprint=self.assert_fingerprint,
    )
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\connection.py", line 969, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
        sock=sock,
    ...<8 lines>...
        tls_in_tls=tls_in_tls,
    )
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\util\ssl_.py", line 480, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\util\ssl_.py", line 524, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\apisik\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\ssl.py", line 455, in wrap_socket
    return self.sslsocket_class._create(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        sock=sock,
        ^^^^^^^^^^
    ...<5 lines>...
        session=session
        ^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\apisik\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\ssl.py", line 1076, in _create
    self.do_handshake()
    ~~~~~~~~~~~~~~~~~^^
  File "C:\Users\apisik\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\ssl.py", line 1372, in do_handshake
    self._sslobj.do_handshake()
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Basic Constraints of CA cert not marked critical (_ssl.c:1028)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\connectionpool.py", line 488, in _make_request
    raise new_e
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Basic Constraints of CA cert not marked critical (_ssl.c:1028)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\requests\adapters.py", line 644, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\connectionpool.py", line 841, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\urllib3\util\retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Basic Constraints of CA cert not marked critical (_ssl.c:1028)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\recreate.py", line 6, in <module>
    r = requests.get("https://google.com")
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\requests\api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\apisik\Home\Dev\req-troubleshoot\.venv-313\Lib\site-packages\requests\adapters.py", line 675, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Basic Constraints of CA cert not marked critical (_ssl.c:1028)')))

First Edit:

Doing some more investigating, it appears if I downgrade urllib3 from version 2.4.0 -> 2.3.0 it starts working.

Maybe seeing what changed between the two might give better insight into what is happening.


Second Edit:

I did the diffing and it appears in python 3.13 stricter verification was added see below:

added to src/urllib3/util/ssl_.py

+    if verify_flags is None:
+        verify_flags = 0
+        # In Python 3.13+ ssl.create_default_context() sets VERIFY_X509_PARTIAL_CHAIN
+        # and VERIFY_X509_STRICT so we do the same
+        if sys.version_info >= (3, 13):
+            verify_flags |= VERIFY_X509_PARTIAL_CHAIN
+            verify_flags |= VERIFY_X509_STRICT
+
+    context.verify_flags |= verify_flags

Third Edit

It appears to be a me problem. Upon further investigation our corporate root certificates may be misconfigured?

What I'm seeing in our certificate when I output it as text using openssl x509 -in cert.pem -text -noout

        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:TRUE

should be... ? (I ask this because I am unsure)

        X509v3 extensions: 
            X509v3 Basic Constraints: critical
                CA:TRUE

apisik avatar Nov 21 '25 19:11 apisik

@apisik that's correct, this is a change in CPython for 3.13 that urllib3 is using. The issue you're encountering isn't directly related to Requests. You can find one of the posts on it here.

You can find some examples there for what typical certificates using this extension look like.

nateprewitt avatar Nov 24 '25 02:11 nateprewitt

@nateprewitt Thanks for confirming!

apisik avatar Nov 24 '25 15:11 apisik

The container image for Apache Airflow with version 3.1.3-python3.12 is including these versions of the packages and thus resulting in such error messages.

This issue might be related to https://github.com/psf/requests/issues/6900

n-badtke-cg avatar Nov 27 '25 16:11 n-badtke-cg