Asyncio integration only instruments one Asyncio event loop

[szokeasaurusrex]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.30.0

Steps to Reproduce

To observe this error, we can add a print statement within the inner _coro_creating_hub_and_span async function (defined within _sentry_task_factory which is itself defined within patch_asyncio) in the file sentry_sdk/integrations/asyncio.py. This inner function is supposed to wrap every async call when the AsyncioIntegration is enabled, and so adding the print statement allows us to verify that asyncio has been correctly instrumented.

Next, we run the following sample script:

import asyncio
import time

import sentry_sdk
from sentry_sdk.integrations.asyncio import AsyncioIntegration


async def fib(n):
    if n == 0:
        return 0

    if n == 1:
        return 1

    time.sleep(0.05)

    async with asyncio.TaskGroup() as tg:
        fib1 = tg.create_task(fib(n - 1))
        fib2 = tg.create_task(fib(n - 2))

    return fib1.result() + fib2.result()


async def main():
    sentry_sdk.init(
        dsn="<dsn here>",
        traces_sample_rate=1.0,
        integrations=(AsyncioIntegration(),),
    )

    await runner()  # This call is instrumented


async def runner():
    with sentry_sdk.start_transaction(op="test", name="Fibonacci"):
        result = await fib(5)

    print(result)


if __name__ == "__main__":
    print("initial call")
    asyncio.run(main())
    print("initial call has ended\n")

    print("second call")
    asyncio.run(runner())  # This call is NOT instrumented
    print("second call has ended\n")

Expected Result

Both calls to runner should be auto-instrumented by the SDK's asyncio integration.

Actual Result

The SDK only auto-instruments the first call to runner, which is made within the main function. The SDK fails to instrument the second call to runner, which is made outside the main function.

Likely Cause

This issue likely has the same cause as #2328; that is to say, this issue appears to be caused by the Sentry SDK only instrumenting the currently running asyncio event loop. asyncio.run() always creates a new event loop every time it is called, which is why the second call to runner fails.

Potential Solutions

1. Don't fix, and close the issue Calling asyncio.run() multiple times, while technically possible, is likely not a common use case, and the Python docs recommend avoiding multiple calls to the function from the same thread. From the docs, "[asyncio.run()] should be used as a main entry point for asyncio programs, and should ideally only be called once." If we go this route, perhaps we should document only supporting a single asyncio.run() call as a known SDK limitation.
2. Modify the SDK to instrument newly created event loops This solution would also likely solve the related issue #2328. And, even if calling asyncio.run() multiple times is not a common or even a recommended design pattern, it is possible to do and so we should support it.

If we go with second option, this issue could likely be resolved by instrumenting asyncio through the use of a custom event loop policy which instruments the SDK when a new event loop is created in addition to instrumenting the current event loop.

[dPeS]

+1 I'm eager to test sentry apm in ray/asyncio...

[antonpirker]

A quick question @dPeS does ray with asyncion call asyncio.run() multiple times?

[antonpirker]

Because having multiple event loops is not recommended (because after the first event loop is closed it could happen, that not everything is cleaned up and data is leaked to the second event loop)

We will not support this, so I am closing this issue.