Missliding warning in aiohttp.connector.TCPConnector in Python 3.11 using uvloop with local proxy via https

[alex-ber]

Describe the bug

See also https://github.com/aio-libs/aiohttp/issues/10676

I'm using Fiddler as local Proxy Server

This code is in some 3rd-party library:

TAVILY_API_URL = "https://api.tavily.com"

async with aiohttp.ClientSession() as session:
    async with session.post(f"{TAVILY_API_URL}/search", json=params) as res:
        if res.status == 200:
            data = await res.text()
            return data
        else:
            raise Exception(f"Error {res.status}: {res.reason}")

.env

http_proxy=http://172.30.176.1:8888
https_proxy=http://172.30.176.1:8888
all_proxy=http://172.30.176.1:8888
SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

I'm using FastAPI on Python 3.11 that by default installs uvloop (not asyncio's event-loop). uvloop's underlying_transport doesn't have_start_tls_compatible attribute, but it does have start_tls.

In order to make this code work with proxy I have to do monkey-patch:

import aiohttp

# monkey-patch
# Save the original ClientSession class
OriginalClientSession = aiohttp.ClientSession


# Define a patched ClientSession that forces the proxy
class PatchedClientSession(OriginalClientSession):
    def __init__(self, *args, **kwargs):
        # Always set trust_env=True to honor proxy environment variables
        kwargs['trust_env'] = True
        super().__init__(*args, **kwargs)


# Apply the patch
aiohttp.ClientSession = PatchedClientSession

Again, I can't change the code above, it is not under my control. Why trust_env has False as default value in the first place? httpx, requests has True by default..._I have created seperated bug https://github.com/aio-libs/aiohttp/issues/10682 for it.

The problematic code is in aiohttp.connector.TCPConnector class mostly in: _create_proxy_connection() functions.

In my case req.is_ssl() is True (and runtime_has_start_tls is also True), so:

self._warn_about_tls_in_tls(transport, req) is called and misleading warning is printed.

Work-arround:

warnings.filterwarnings(
        'ignore',
        category=RuntimeWarning,
        message=r".*HTTPS request.*through an HTTPS proxy.*TLS\s+in\s+TLS.*"
    )

Suggested fix:

Alternative 1: Introducing separate warning category making suppressing warning easier. Alternative 2: Rewriting self._warn_about_tls_in_tls(transport, req) taking into account other than asyncio's event-loop.

Python Version

$ python --version
Python 3.11.11

aiohttp Version

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.11.9
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author:
Author-email:
License: Apache-2.0
Location: /usr/local/lib/python3.11/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, propcache, yarl
Required-by: aiobotocore, langchain, langchain-community

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 6.1.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /usr/local/lib/python3.11/site-packages
Requires:
Required-by: aiohttp, yarl

propcache Version

$ python -m pip show propcache
Name: propcache
Version: 0.2.1
Summary: Accelerated property cache
Home-page: https://github.com/aio-libs/propcache
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache-2.0
Location: /usr/local/lib/python3.11/site-packages
Requires:
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache-2.0
Location: /usr/local/lib/python3.11/site-packages
Requires: idna, multidict, propcache
Required-by: aiohttp

OS

python:3.11-slim

Related component

Client

Additional context

No response

Code of Conduct

• [x] I agree to follow the aio-libs Code of Conduct