test suite gets really slow in pypy3.9-v7.3.15 on ubuntu 22.04.3

[ecerulm]

Describe the bug

In urllib3, we just downgraded to pypy3.9-v7.3.13 (https://github.com/urllib3/urllib3/pull/3308) because the test suite takes more that 30 minutes when run on pypy3.9-v7.3.15 (we don't know how long exactly because the CI jobs is cancelled at the 30 minutes mark) and less than 10 minutes when run on pypy3.9-v7.3.15.

To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

1. What version of Python are you using? pyp3.9-v7.3.15 on Ubuntu 22.04.3 LTS
2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful. 7.4.0
3. What versions of what packages do you have installed? The output of pip freeze is helpful.

aiofiles==23.2.1
anyio==4.2.0
attrs==23.2.0
blinker==1.7.0
brotlicffi==1.1.0.0
certifi==2023.11.17
cffi==1.16.0
click==8.1.7
click-default-group==1.2.4
coverage==7.4.0
cryptography==41.0.6
exceptiongroup==1.2.0
Flask==3.0.1
greenlet==0.4.13
h11==0.14.0
h2==4.1.0
hpack==4.0.0
hpy==0.9.0
httpcore==1.0.2
httpx==0.25.2
Hypercorn @ git+https://github.com/urllib3/hypercorn@d1719f8c1570cbd8e6a3719ffdb14a4d72880abb
hyperframe==6.0.1
idna==3.4
importlib-metadata==7.0.1
importlib-resources==6.1.1
incremental==22.10.0
iniconfig==2.0.0
itsdangerous==2.1.2
Jinja2==3.1.3
MarkupSafe==2.1.4
outcome==1.3.0.post0
packaging==23.2
pluggy==1.3.0
priority==2.0.0
pyOpenSSL==23.2.0
PySocks==1.7.1
pytest==7.4.2
pytest-timeout==2.1.0
Quart==0.19.4
quart-trio==0.11.1
readline==6.2.4.1
sniffio==1.3.0
sortedcontainers==2.4.0
taskgroup==0.0.0a4
tomli==2.0.1
towncrier==23.6.0
trio==0.23.1
trustme==1.1.0
typing_extensions==4.9.0
urllib3
Werkzeug==3.0.1
wsproto==1.2.0
zipp==3.17.0
zstandard==0.22.0

1. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

The workaround if was to use pypy3.9-v7.3.13 (https://github.com/urllib3/urllib3/pull/3308/commits/f14b50840f36dd1da235accedd4b71e84b7eb7b8)

The version of the urllib3 that still shows the problem in CI is https://github.com/urllib3/urllib3/commit/6b2b377d9acc996f3cd8296af3b5d5cf7a255a09

1. What commands should we run to reproduce the problem? Be specific. Include everything, even git clone, pip install, and so on. Explain like we're five!

I'll expand later because I can't reproduce this myself on my laptop (macOS Sonoma pypy3.9-v7.3.15) so I think it maybe only reproducible on Ubuntu 22.04.3 LTS

Expected behavior

I expected it to complete the test suite (with coverage) in under 10 minutes while using pypy3.9-v7.3.15 as it does when using pypy3.9-v7.3.13

Additional context

[ecerulm]

Examples of slow test suite execution under coverage for pypy3.9-v7.3.15

https://github.com/urllib3/urllib3/actions/runs/7603721375/job/20705793502?pr=3296 https://github.com/urllib3/urllib3/actions/runs/7559660055/job/20583952938?pr=3275 https://github.com/urllib3/urllib3/actions/runs/7615341858/job/20739660497?pr=3273

they all stop at the 30 minutes mark (the CI jobs get cancelled after 30 minutes) but they show that it's not stuck in any particular test case , all of them progress to a different number of test cases.

I haven't been able to reproduce it in macOS , I'll later try on Ubuntu 22.04 (outside of CI) and post the findings here.

[ecerulm]

I performed a test with pypy3.9-7.3.15 and without coverage and it still takes more that 30 minutes (although it progresses faster that with coverage), so I guess this is probably an issue with pypy3.9-7.3.15 and not really related to coverage.

[ecerulm]

https://github.com/pypy/pypy/issues/4877

[nedbat]

Thanks, we have our own issue with 7.3.15: #1737 . I assume it's the same root cause, whatever it is. For us, it's only on Windows.

[devdanzin]

I performed a test with pypy3.9-7.3.15 and without coverage and it still takes more that 30 minutes (although it progresses faster that with coverage), so I guess this is probably an issue with pypy3.9-7.3.15 and not really related to coverage.

I think this can be closed then. Everything points to a performance issue with OpenSSL 3.0 being the culprit (https://github.com/pypy/pypy/issues/4877).