HTTPS proxy certificate is not validated

[arossert]

When running a request using HTTPS proxy the proxy certificate is ignored even when verify=True. Is there a way to validate the proxy certificate?

>>> requests.get("https://www.google.com", proxies={"https": "https://100.117.2.81:3129"}, verify=True)
>>> <Response [200]>

This is available using pycurl

curl.setopt(pycurl.PROXY_SSL_VERIFYPEER, True)
curl.setopt(pycurl.PROXY_SSL_VERIFYHOST, True)

[hasanozdem1r]

Hello @arossert If you have a copy of the self-signed certificate and key you can modify the code as follow:

proxies = {
    'http': 'http://127.0.0.1:24000',
    'https': 'http://127.0.0.1:24000',
}

certificate_path = os.path.join(CACERT_PATH, 'cacert.pem')
key_path = os.path.join(CACERT_KEY, 'cacert.key')

resp = requests.get('https://api.myip.com',
                    proxies=proxies,
                    cert=(certificate_path, key_path))
print(resp.text)

[arossert]

@hasanozdem1r This will verify the target host https://api.myip.com, what I'm interested in is verifying the proxy server.

proxies = {'http': 'https://mp.proxy.com:1234', 'https': 'https://my.proxy.com:1234'}
resp = requests.get('https://api.myip.com', proxies=proxies)

So in this case I want to be able to validate my.proxy.com certificate, it is possible using curl/pycurl

[hasanozdem1r]

@arossert well I think you should make it clear with curl/ pycurl part. Following code you've mentioned will verify proxy server already.

proxies = {'http': 'https://mp.proxy.com:1234', 'https': 'https://my.proxy.com:1234'}
resp = requests.get('https://api.myip.com', proxies=proxies)

Even though here is more better approach

def check_proxy(proxy, timeout, list, proxy_type):
        global working
        global dead
        try:
            try:
                proxies = {
                    'http': f'{proxy_type}://{proxy}',
                    'https': f'{proxy_type}://{proxy}'
                }
                request.get("https://www.google.com/", timeout=timeout, proxies=proxies)
            except Exception:
                printqueue.append(f"[{proxy}]: Dead or timed out.")
                dead += 1
            else:
                printqueue.append(f"[{proxy}]: Working")
                working += 1
                name = list.split("/")[-1].split(".")[0]
                with open(f"proxies/{name}-{proxy_type}-working.txt", "a+") as handle:
                    handle.write(f"{proxy}\n")
        except Exception as e:
            printqueue.append(e) 

Do you want to pass from curl proxies and verify or how ? Here also way to do that is explained how to verify proxy from curl https://oxylabs.io/blog/curl-with-proxy

[arossert]

@hasanozdem1r Maybe I was not clear enough, but I want to verify the proxy server certificate, not that the proxy is working.

When working with HTTPS, there is a certificate validation step that requests is skipping on the proxy server even when verify=True.

Here is an example Curl

curl --proxy https://100.117.2.81:3129 https://www.google.com
curl: (60) SSL: no alternative certificate subject name matches target host name '100.117.2.81'
(can be skipped with --proxy-insecure)

Request

import requests
requests.get("https://www.google.com", proxies={"https": "https://100.117.2.81:3129"}, verify=True)
<Response [200]>

I expect the request to fail on a proxy certificate validation error.