requests icon indicating copy to clipboard operation
requests copied to clipboard

Wrong CA Certificates used in requests.Session.send

Open sunnstix opened this issue 1 year ago • 1 comments

Bug in requests.sessions.py:

https://github.com/psf/requests/blob/b8be93a721792deeadd2f498b8f77cf610e7765f/src/requests/sessions.py#L579-L589

session.merge_environment_settings is called in session.request and not in session.send which means if users want to build their own prepared requests (see example below), the correct certificates file will not be used unless explicitly passed to send.

I believe these environment settings should be updated in session.send instead.

Reproduction Steps

import os
import requests

# Set CA certificates file to custom local file
os.environ["REQUESTS_CA_BUNDLE"] = "/etc/ssl/certs/ca-certificates.crt"

# Create request object
req = requests.Request(
    method="GET", # any method
    url="https://www.example_site.mydomain.com/path" # any url for which the default cert file does not have a cert for
)

with requests.Session() as s:
    respA = s.request( # Success: will use CA certificates from REQUESTS_CA_BUNDLE
        method=req.method,
        url=req.url
    )
    assert respA.status_code == 200
    
with requests.Session() as s:
    respB = s.send( # Success: will use CA certificates from REQUESTS_CA_BUNDLE if explicitly specified
        request=req.prepare(), verify=os.environ["REQUESTS_CA_BUNDLE"]
    )
    
    assert respB.status_code == 200
    
with requests.Session() as s:
    respC = s.send( # Error: Will try to use default CA certificates instead of those specified by REQUESTS_CA_BUNDLE
        request=req.prepare() 
    )
    
    assert respC.status_code == 200

Expected Result

No output

Actual Result

requests.exceptions.SSLError: HTTPSConnectionPool(host='https://www.example_site.mydomain.com/path', port=443): Max retries exceeded with url: /path (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))

System Information

$ python -m requests.help
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.4"
  },
  "platform": {
    "release": "5.4.0-150-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30000020"
  },
  "urllib3": {
    "version": "2.2.1"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

sunnstix avatar Feb 23 '24 08:02 sunnstix

I can confirm I'm seeing this behavior when using prepared requests and sessions.send(), being forced to set session.verify = os.environ['REQUESTS_CA_BUNDLE'] before send().

bmeneg avatar May 30 '24 21:05 bmeneg