Support Python 3.11

[Yun-Kim]

This issue is to track our progress in supporting Python 3.11.

The dd-trace-py library currently does not support Python 3.11, which is scheduled to be released October 24 2022. We have several integrations and core components that are failing to build or failing tests with #4125.

Checklist for failing dd-trace-py core components:

1. ❌ ddtrace-run: gevent dependency fails to install due to API changes to their vendored greenlet.

• Waiting on gevent to unblock and support 3.11 issue and PR.

2. ❌ internal & debugger: depend on the bytecode module which doesn't support Python 3.11 yet, so the debugger and internal test suites won't pass.
3. ❌ profiler: see #4125 description, however the profiler is still crashing on Python 3.11.
4. ✅ tracer: ~Riot seems to have problems with Python 3.11's importlib in terms of managing virtual environments. Specifically, Hypothesis' vendored text file is not being found even though there is no problem running the test locally with Python 3.11.~

• Workaround was to ignore the failing test file that imported from hypothesis.provisional, and create a separate test suite using tox to test it separately.

Checklist for failing integrations:

1. ❌ celery: getting a AttributeError: '_Code' object has no attribute 'co_positions' error for 2 celery tests, failing tests marked as flaky but they are constantly failing with PR #4125

• Skipping testing with Python 3.11 for now as celery's dependency billiard is not yet Python 3.11 compatible. See issue

2. ✅ cherrypy & pynamodb: ~inspect.getargspec() has been deprecated since Python 3.0 and was now removed in Python 3.11.~

• ~Known issue and resolved by cherrypy fix, released in version 18.7.~
• ~In other words Python 3.11 is only compatible with Cherrypy >= 18.7.~
• ~Pynamodb released compatibility fix for removed getargspec() in version 4.1~
• Updated cherrypy & pynamodb test suites to use versions 18.7 & 4.1 respectively with Python 3.11.

3. ✅ dogpilecache: ~inspect.getargspec() has been deprecated since Python 3.0 and was now removed in Python 3.11.~

• ~Known issue and resolved in dogpilecache >= 0.7~
• Updated dogpilecache test suite to run with Py 3.11 with dogpilecache >= 0.7.

6. ✅ kombu: ~failing in CI due to a pinned dependency.~

• ~Kombu's dependency Vine==1.3.0 is pinned in CI for some reason and it is incompatible with Python 3.11 (inspect.formatargspec() has been deprecated since Python 3.5 and was now removed in Python 3.11.)~
• kombu >= 5.0 is compatible with Python 3.11, updated kombu test suite to run with kombu >= 5.0

7. ✅ django: ~Python's gettext.py module removed a deprecated function argument codeset as of Python 3.11.~

• ~Django 4.1 is meant to be first django version compatible with Python 3.11~
• Updated django test suite to run Py3.11 with Django 4.1.

8. ❌ fastapi & starlette:

• Getting TypeError: Passing coroutines is forbidden, use tasks explicitly. errors from asyncio
• fastapi giving the same error, makes sense as it is based on starlette

9. ❌ graphene, graphql:

• Getting error as they are also dependent on the bytecode module, which doesn't yet support Python 3.11

10. ❌ grpc:

• ~According to grpc maintainers grpc 1.49.0 will support python 3.11 which should be released soon.~
• Updated grpc test suites to use 1.49.0 with Python 3.11.
• grpc test suite with asyncio and python 3.11 are still failing.

11. ✅ mariabd: ~fails to install with Python 3.11 currently, version 1.1.2 onwards should be compatible with Python 3.11.~

• Updated mariadb test suite to use 1.1.2+ for Python 3.11.

12. ❌ asyncpg: getting asyncpg/pgproto/pgproto.c:223:12: fatal error: longintrepr.h: No such file or directory #include "longintrepr.h" error during install process

• Should be fixed in coming release of asyncpg. See issue
• Skipping the asyncpg python 3.11 tests for now until asyncpg releases a compatible version.

13. ✅ aioredis: ~failing to run test cases with TypeError: duplicate base class "TimeoutError"~

• According to this aioredis issue, aioredis is being replaced by the redis library (4.2.0+), and no longer supports Python 3.11 (redis will take over support for aioredis). Therefore, the aioredis test suite has been update to exclude Python 3.11 and an internal issue has been raised to transition our integration/testing for aioredis into our redis integration.

14. ✅ psycopg: ~failing to install.~

• ~According to psycopg docs, version 2.9.2 has preliminary support for python 3.11, but our riotfile tests are based on ~=2..8.0, ~=2.9.0, latest~
• Updated psycopg test suite to use 2.9.2+ with Python 3.11.

15. ✅ pylibmc: ~fails to build with Python 3.11, fixed in pylibmc==1.6.1~

• Updated pylibmc test suite to use 1.6.1+ with Python 3.11.

[Yun-Kim]

Currently running into a problem with gevent failing to build properly with Python 3.11. I suspect it has to do with gevent's vendored version of greenlet, and I've made an issue on gevent's github page here.

[Yun-Kim]

Note that I've separated the profiler changes into a separate PR #4343 so that it can unblock the tracer PR #4125.

[fukszssl247]

docker python just updated to 3.11 wheel fails to build if used with python 3 latest

[Yun-Kim]

Status Update: The tracer has preliminary support for Python 3.11 as of version 1.6.0. This unfortunately does not yet include integrations for asyncpg, graphene, graphql, celery, and grpcio with pytest-asyncio, as well as the profiler and dynamic instrumentation.