Consider using super() to initialize httpx clients

[Lawouach]

Describe the bug

Hi,

I believe there may be an issue when mixing this library with opentelemetry-instrumentation-httpx.

Until recently all was well, but today I suddenly had a failure:

File "venv/lib/python3.11/site-packages/reliably_app/login/service.py", line 138, in login_with_provider
    response = await client.authorize_redirect(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/starlette_client/apps.py", line 34, in authorize_redirect
    rv = await self.create_authorization_url(redirect_uri, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/base_client/async_app.py", line 103, in create_authorization_url
    async with self._get_oauth_client(**metadata) as client:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/base_client/sync_app.py", line 215, in _get_oauth_client
    session = self.client_cls(
              ^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/httpx_client/oauth2_client.py", line 65, in __init__
    httpx.AsyncClient.__init__(self, **client_kwargs)
  File ".venv/lib/python3.11/site-packages/opentelemetry/instrumentation/httpx/__init__.py", line 483, in __init__
    super().__init__(*args, **kwargs)
    ^^^^^^^
TypeError: super(type, obj): obj must be an instance or subtype of type"

When you look at how the instrumentation lib works, you can see they subclass httpx.AsyncClient:

https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/init.py#L483C9-L483C42

They use the super keyword. However, authlib does use the "older" mechanism to initialize its subclass https://github.com/lepture/authlib/blob/master/authlib/integrations/httpx_client/oauth2_client.py#L65

I wonder if that means the tree isn't properly constructed by Python 3.11 which leads to this error.

Error Stacks

File "venv/lib/python3.11/site-packages/reliably_app/login/service.py", line 138, in login_with_provider
    response = await client.authorize_redirect(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/starlette_client/apps.py", line 34, in authorize_redirect
    rv = await self.create_authorization_url(redirect_uri, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/base_client/async_app.py", line 103, in create_authorization_url
    async with self._get_oauth_client(**metadata) as client:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/base_client/sync_app.py", line 215, in _get_oauth_client
    session = self.client_cls(
              ^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/authlib/integrations/httpx_client/oauth2_client.py", line 65, in __init__
    httpx.AsyncClient.__init__(self, **client_kwargs)
  File ".venv/lib/python3.11/site-packages/opentelemetry/instrumentation/httpx/__init__.py", line 483, in __init__
    super().__init__(*args, **kwargs)
    ^^^^^^^
TypeError: super(type, obj): obj must be an instance or subtype of type"

To Reproduce

It's not trivial to set a basic example. I'll do my best to update accordingly.

Expected behavior

A clear and concise description of what you expected to happen.

Environment:

• Ubuntu
• Python 3.11.2
• authlib 1.2.1
• opentelemetry-instrumentation-httpx 0.39b0
• httpx 0.24.1

[lspgn]

Same issue :( . My current workaround is binding authlib==1.2.0

[lepture]

@Lawouach @lspgn A pull request is welcome.

[lspgn]

@lepture @Lawouach btw thank you for your work!

I tried as much as I could to fix it but I don't really know how. I am suspecting it could be a bug in httpx as well. Sorry :( Was only able to make a script to reproduce and tried to get as much info as I could:

#!/usr/bin/env python3

from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor

from starlette.config import Config
from authlib.integrations.starlette_client import OAuth

import jwt
from jwt import PyJWKClient

import asyncio

HTTPXClientInstrumentor().instrument() # works without this line

config = Config(".env")
oauth = OAuth(config)

oauth_base = "https://accounts.google.com"
auth0 = oauth.register(
    "test",
    server_metadata_url="{}/.well-known/openid-configuration".format(oauth_base),
)

jwks_client = None

async def lifespan_oauth():
    global jwks_client
    if oauth_base:
        oauth_metadata = await auth0.load_server_metadata()
        jwks_client = PyJWKClient(oauth_metadata.get("jwks_uri"))

event_loop = asyncio.get_event_loop()
future = asyncio.ensure_future(lifespan_oauth(), loop=event_loop)

event_loop.run_until_complete(future)
event_loop.close()

print("ran without issues")

and the requirements.txt

Authlib==1.2.1
opentelemetry-exporter-otlp-proto-common==1.19.0
opentelemetry-instrumentation==0.40b0
opentelemetry-instrumentation-asgi==0.40b0
opentelemetry-proto==1.19.0
opentelemetry-semantic-conventions==0.40b0
opentelemetry-util-http==0.40b0
opentelemetry-exporter-otlp-proto-http==1.19.0
opentelemetry-instrumentation-logging==0.40b0
opentelemetry-instrumentation-httpx==0.40b0
opentelemetry-instrumentation-sqlalchemy==0.40b0
starlette==0.27.0
httpx==0.24.1
PyJWT==2.8.0
## The following requirements were added by pip freeze:
anyio==3.7.1
asgiref==3.7.2
backoff==2.2.1
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
cryptography==41.0.3
Deprecated==1.2.14
googleapis-common-protos==1.60.0
h11==0.14.0
httpcore==0.17.3
idna==3.4
importlib-metadata==6.8.0
opentelemetry-api==1.19.0
opentelemetry-sdk==1.19.0
packaging==23.1
protobuf==4.23.4
pycparser==2.21
requests==2.31.0
sniffio==1.3.0
typing_extensions==4.7.1
urllib3==2.0.4
wrapt==1.15.0
zipp==3.16.2

Source of the issue:

https://github.com/open-telemetry/opentelemetry-python-contrib/blob/7603a1fc69474398289c2944796249e70bba0c82/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/init.py#L531

With v1.2.1: in oauth2_client.py line 65, httpx.AsyncClient has the following value: opentelemetry.instrumentation.httpx._InstrumentedAsyncClient (vs httpx.AsyncClient otherwise)

If I change line 65:

super(httpx.AsyncClient).__init__( **client_kwargs)

Suddenly it complains about _state:

    if self._state != ClientState.UNOPENED:
       ^^^^^^^^^^^
AttributeError: 'AsyncOAuth2Client' object has no attribute '_state'. Did you mean: 'state'?

Not sure if it's in the correct direction.

[staticdev]

@lepture do you have an insight what would be the fix here? It would be great if we can have latest authlib with otel instrumentation working good.