Segmentation fault in starlette integration after upgrade to Python 3.12
How do you use Sentry?
Self-hosted/on-premise
Version
1.45.0
Steps to Reproduce
After upgrading from Python 3.10.13 to 3.12.4 we started getting segmentation faults from time to time in a service. As soon as 3.12.5 was released we upgraded to it and it didn't help. The problem is not easily reproducible, we have to wait for hours to get it in heavy-loaded service.
Expected Result
No segmentation fault.
Actual Result
The stack trace (as we got by setting PYTHONFAULTHANDLER environment variable) for the current thread is (for 3.12.5, only line numbers differ for 3.12.4):
Current thread 0x00007f155df85b80 (most recent call first):
File "/user/.local/lib/python3.12/site-packages/sentry_sdk/integrations/starlette.py", line 137 in _sentry_receive
File "/user/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 54 in wrapped_receive
File "/user/.local/lib/python3.12/site-packages/starlette/responses.py", line 238 in listen_for_disconnect
File "/user/.local/lib/python3.12/site-packages/starlette/responses.py", line 261 in wrap
File "/user/.local/lib/python3.12/site-packages/starlette/responses.py", line 265 in __call__
File "/user/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 192 in __call__
File "/user/.local/lib/python3.12/site-packages/sentry_sdk/integrations/starlette.py", line 158 in _create_span_call
File "/user/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 151 in coro
File "/usr/local/lib/python3.12/asyncio/runners.py", line 118 in run
File "/usr/local/lib/python3.12/asyncio/runners.py", line 194 in run
File "/user/.local/lib/python3.12/site-packages/uvicorn/server.py", line 65 in run
File "/user/.local/lib/python3.12/site-packages/uvicorn/main.py", line 577 in run
File "/user/service/app/server.py", line 98 in server
File "/user/.local/lib/python3.12/site-packages/click/core.py", line 783 in invoke
File "/user/.local/lib/python3.12/site-packages/click/core.py", line 1434 in invoke
File "/user/.local/lib/python3.12/site-packages/click/core.py", line 1688 in invoke
File "/user/.local/lib/python3.12/site-packages/click/core.py", line 1078 in main
File "/user/.local/lib/python3.12/site-packages/click/core.py", line 1157 in __call__
File "/user/service/app/main.py", line 57 in <module>
File "<frozen runpy>", line 88 in _run_code
File "<frozen runpy>", line 198 in _run_module_as_main
Extension modules: greenlet._greenlet, sqlalchemy.cyextension.collections, sqlalchemy.cyextension.immutabledict, sqlalchemy.cyextension.processors, sqlalchemy.cyextension.resultproxy, sqlalchemy.cyextension.util, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, _cffi_backend, asyncpg.pgproto.pgproto, asyncpg.protocol.protocol, markupsafe._speedups, uvloop.loop (total: 14)
Not sure if Sentry is to blame, but maybe you could point me in the right direction.
Hey @ods ,
the only thing that comes to my mind after looking at the stack trace is trying to disable the middleware spans in the Starletteintegration. See the docs here: https://docs.sentry.io/platforms/python/integrations/starlette/#options
And then try again.
I know this is no solution, but maybe we can narrow it down to the middleware span generation.
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!
"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀