Modules Integration Blocking Run Loop with ASGI Integration

[ejlangev]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.28.1

Steps to Reproduce

Initialize the sentry SDK in a blank FastAPI application. Enable the following integrations:

FastApiIntegration(transaction_style="url"),
HttpxIntegration(),
LoggingIntegration(level=logging.INFO, event_level=logging.FATAL),
RedisIntegration(),
SqlalchemyIntegration(),

The first time you call report_exception the modules integration will load and cache all the modules you have for reporting purposes. Doing this can actually be fairly slow and causes run loop blocking which delays other requests. The call stack in question here is the following:

__call__ (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/fastapi/applications.py:282)
_sentry_patched_asgi_app (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:348)
_run_asgi3 (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:141)
_run_app (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:186)
_capture_exception (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:62)
capture_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/hub.py:350)
capture_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/client.py:522)
_prepare_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/client.py:284)
wrapper (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/scope.py:77)
apply_to_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/scope.py:647)
processor (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/modules.py:76)
_get_installed_modules (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/modules.py:57)
_generate_installed_modules (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/modules.py:32)
metadata (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/importlib_metadata/__init__.py:455)
read_text (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/importlib_metadata/__init__.py:837)
read_text (/root/.pyenv/versions/3.10.11/lib/python3.10/pathlib.py:1134)
open (/root/.pyenv/versions/3.10.11/lib/python3.10/pathlib.py:1119)
open64 (/lib/x86_64-linux-gnu/libc-2.31.so:0)

The problem is that within the async def _run_app method of the ASGI middleware the capture_exception method which ends up directly calling _generate_installed_modules is synchronous so any IO it does directly will block the main run loop and prevent other coroutines from running.

To fix it should either run in a thread or not do any blocking IO. If the latter I think it would need to provide a way of preloading the modules at initialization time for example.

Expected Result

Run loop is not blocked when loading modules

Actual Result

Run loop is blocked when loading modules

[antonpirker]

Hey @ejlangev

Thanks for reporting this! But it on our (big) list of things to fix. Will take some time though. But PRs are always welcome, if you want to give fixing it a try!

[antonpirker]

Hey @ejlangev !

It has been a while, but in November 2023 with version 1.37.0 we released a refactoring of _generate_installed_modules.

This change should speed up every call to _generate_installed_modules after the first one considerably.

Can you check if this solves your problem and makes your FastAPI application not blocking the run loop?

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀