When initialized before FastAPI app, sentry doesn't capture errors

[janluke]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.30.0

Steps to Reproduce

With the following code, I don't get any error captured:

import logging

import sentry_sdk
from asgi_correlation_id import CorrelationIdMiddleware
from fastapi import FastAPI
from starlette.middleware.gzip import GZipMiddleware

from app_logging import configure_logging
from config import settings
from database import Base, engine

configure_logging()
logger = logging.getLogger("app")

Base.metadata.create_all(engine)

sentry_sdk.init(
    settings.sentry.dsn,
    traces_sample_rate=0.0,  # disable performance monitoring
    environment=settings.environment.value,
)

# App
app = FastAPI(title="My API")
app.add_middleware(CorrelationIdMiddleware)
app.add_middleware(GZipMiddleware, minimum_size=1000)

If I move back sentry.init() at the end of this code block, as it was before I read the documentation suggesting to initialize sentry before the app, it works as expected.

I use the [fastapi] extra.

Expected Result

The errors should be captured when initializing sentry before the app. Otherwise the doc should be updated.

Actual Result

No error is captured.

[sentrivana]

Hey @janluke, thank you for reporting this! The docs should be correct (in general, init()ing the SDK as soon as possible should be the way to go) so this indeed looks like an SDK bug.

[yoavkedem1]

@sentrivana Any update on this?

[sentrivana]

@yoavkedem1 No update yet, we still need to look into why the app doesn't get instrumented properly. Does the workaround from the OP work for you in the meantime? (That is, moving the SDK init after the app has been initialized.)

[chgad]

I'm experiencing the same issue and moving the sentry initialization after the FastApi() init does NOT solve it.

I encountered the Issue when a faulty MongoDB connection attempt did not cause any Sentry issues.

[sentrivana]

Couple of questions for folks experiencing this:

• What errors don't get reported? @chgad you mentioned DB errors. If an endpoint throws an exception, is that captured?
• If you init the SDK with debug=True and then an exception happens, can you see the SDK sending it in the debug output? ([sentry] DEBUG: Sending envelope [envelope with 1 items (error)] project:*** host:***)
• What SDK and FastAPI versions are you running?
• Is there any difference in behavior between the dev and prod server?

I used a slimmed down version of @janluke's original example which is working fine for me, meaning I'm either testing something else than y'all or the problematic parts are in code that I don't have access to.

Here's what I did:

import logging
import sentry_sdk
from asgi_correlation_id import CorrelationIdMiddleware
from fastapi import FastAPI
from starlette.middleware.gzip import GZipMiddleware

logger = logging.getLogger("app")

sentry_sdk.init(
    dsn="<dsn>",
    traces_sample_rate=0.0,
)

app = FastAPI(title="My API")
app.add_middleware(CorrelationIdMiddleware)
app.add_middleware(GZipMiddleware, minimum_size=1000)

@app.get("/")
async def root():
    raise ValueError("oh no!")
    return {"message": "Hello World"}

Can y'all try this out and see whether the error gets sent to Sentry?

If someone could compile a small standalone repro example complete with an error that you expect in Sentry but it's not sent that'd be huge help.

[chgad]

What errors don't get reported? @chgad you mentioned DB errors. If an endpoint throws an exception, is that captured?

Within an endpoint i tried connecting to a, at this point non-existing database. Pymongo raised an pymongo.errors.OperationFailure error. Explicitly raising an Exception() within a view did not send it at all.

import sentry_sdk
from fastapi import FastAPI

from .settings import get_settings

SETTINGS = get_settings()

if SETTINGS.DEBUG:
    sentry_sdk.init(dsn=SETTINGS.SENTRY_URL, enable_tracing=True, debug=True)

app = FastAPI()


@app.get("/trigger-error")
def trigger_error():
    raise ValueError("Invalid Error")
    return {}

Trying the code above does not capture the ValueError() and produces the following logs:

2024-06-19T14:11:09.310448412Z app[web.1]:  [sentry] INFO: event processor (<function DedupeIntegration.setup_once.<locals>.processor at 0x7fc4d51af2e0>) dropped event
2024-06-19T14:11:09.310468477Z app[web.1]:  [sentry] DEBUG: Sending envelope [envelope with 2 items (error, internal)] project:XX host:XX.XXXXXXX.com

(X's being correct id and host.)

Furthermore there is another log, which shows this (truncated for readability:

                <div class="box-content with-padding">
                    <section class="body">
                        <div class="page-header"><h2>CSRF Verification Failed</h2></div>
                        <p>A required security token was not found or was invalid.</p>
                        <p>If you\'re continually seeing this issue, try the following:</p>
                        <ol>
                            <li>Clear cookies (at least for Sentry\'s domain).</li>
                            <li>Reload the page you\'re trying to submit (don\'t re-submit data).</li>
                            <li>Re-enter the information, and submit the form again.</li>
                        </ol>
                        <p>You are seeing this message because Sentry requires a \'Referer header\' to be sent by
                            your Web browser, but none was sent. This header is required for security reasons, to
                            ensure that your browser is not being hijacked by third parties.</p>
                        <p>If you have configured your browser to disable \'Referer\' headers, please re-enable
                            them, at least for this site, or for HTTPS connections, or for \'same-origin\'
                            requests.</p>
                        <p>Read more about <a href="http://en.wikipedia.org/wiki/Cross-site_request_forgery">CSRF on
                            Wikipedia</a>.</p></section>
                </div>

Where did i forget to add a Referer Header? Doc's dont tell me to add this anywhere.

EDIT:

I was told just now, that forever reason we are working on a Sentry 9.1 installation... this might be the cause of the error.

[antonpirker]

Hello @chgad ! Can you try to create a project on Sentry.io (its free) and then use that DSN in your app to see if it works with out Saas?

Sentry 9.1 is over five years old, so it could be that the SDK does not work with that version anymore.

[chgad]

Using a project on Sentry.io, everything works as expected. Now it's highly probable that it's the rather antique sentry installation which does not work with the latest sdk. Already working on issueing an update request. Sorry for bothering.

[antonpirker]

No worries @chgad that's what we are here for :-)

For the others in this thread, having problems with FastAPI, please create a new Issue for your specific problem!

I will close this issue

[janluke]

Two notes:

• I had this problem with Sentry Cloud
• this bug is silent and so it's difficult to evaluate its impact: there might be a lot of people that think their errors are being tracked while they are not (unless you really think people keep testing Sentry works after the first time they tested it).

[ThiagoDiasV]

Same issue here. I can only get bugs being caught by Sentry within the FastAPI context, like issues on the endpoint handlers. Before FastAPI context the issues are not being sent to Sentry, even though Sentry SDK is initialized properly.

I have something like this

def init_sentry():
    sentry_sdk.init(
        dsn=<dsn>,
        sample_rate=1.0,
        traces_sample_rate=0.1,
        attach_stacktrace=True,
        environment=<environment>,
        debug=True,
        integrations=[AwsLambdaIntegration()],
    )
    logger.info("Sentry initialized", environment=environment)

def create(container: AppContainer) -> FastAPI:
    app = FastAPI(root_path=API_ROOT_PATH)
    app.add_middleware(
        CORSMiddleware,
        allow_origins=["*"],
        allow_methods=["*"],
        allow_headers=["*"],
        allow_credentials=True,
        expose_headers=["*"],
    )
    app.include_router(router.query_router)
    app.dependency_overrides[deps.get_app_container] = lambda: container
    return app


def lambda_handler(event, context):
    init_sentry()
    logger.info("Handling Lambda event", **event)
    from mangum import Mangum

    container = app_container_factory()
    return Mangum(create(container))(event, context)

If I raise exceptions, for example, on this app_container_factory, Sentry is not sending the envelopes properly. But errors on the FastAPI app, i.e. in some endpoints, are handled properly by the SDK.

Before handling the SDK initialization manually on my code I was using Sentry Layer for my AWS Lambdas and it was working fine.

[andrewtryder]

I've got the same issue on my FastAPI application. It will send messages via capture_message but no exceptions are sent. This is after it was working fine upon setup. Just upgraded to the latest Sentry SDK, too. Is there any other things to try?

Thanks

[sentrivana]

Hey folks, thanks for helping us with more info on this issue. I've reopened this now. I'm still not able to repro -- more repro examples are very much appreciated. Alternatively if you folks can try my example here and tell me if it works for you that's also a valuable data point for us.

@ThiagoDiasV, is there anything suspicious in the logs, e.g. internal errors from the SDK? You might need to set debug=True in the SDK init to see those (caution: more log output). While debug is True, can you also check if you see something like

[sentry] DEBUG: Sending envelope [envelope with 1 items (error)] project:*** host:***

for the exception that doesn't get to Sentry?

@andrewtryder, you say it was working upon setup, what has changed since then? Could you also try setting debug=True and looking for possible internal errors in the logs and seeing whether anything like

[sentry] DEBUG: Sending envelope [envelope with 1 items (error)] project:*** host:***

gets logged?

[mjpieters]

I can see one issue with how the sentry FastAPI handler patches fastapi.routing.get_request_handler, which is a function that's called from all the route decorators used by FastAPI. This issue doesn't explain every issue reported here but may be behind some of the issues.

To illustrate, lets look at this code from the FastAPI integration documentation:

from fastapi import FastAPI

sentry_sdk.init(...)  # same as above

app = FastAPI()

@app.get("/sentry-debug")
async def trigger_error():
    division_by_zero = 1 / 0

Here, it is the @app.get() decorator that will end up calling fastapi.routing.get_request_handler() when Python executes the decorator for trigger_error(). Note that this applies to all such decorators, including those on a fastapi.APIRouter instance.

Now, lets see how this could fail. In the following code, the sentry_sdk.init() call will come too late to instrument the APIRouter-registered routes:

from fastapi import APIRouter, FastAPI

independent_routes = APIRouter()

@independent_routes.get("/sentry-debug")
async def trigger_error():
    division_by_zero = 1 / 0

sentry_sdk.init(...)  # same as above

app = FastAPI()
app.include_router(independent_routes)

The use of APIRouter instances is a common setup in production code, albeit with multiple APIRouter objects spread across multiple modules to help manage large applications. You then use import statements, at the top of the module, to pull in those routes and register them with your FastAPI instance. It is the imports that then trigger route registrations, and imports are usually run before you run code like sentry_sdk.init(). Unfortunately, the Sentry documentation doesn't tell you you need to run the init() call before you register any routes.

This doesn't disable instrumentation entirely, because the Startlette patches also apply and these affect methods that run later (mostly when a request is handled).

[sl0thentr0py]

thanks @mjpieters, that helps.

the fastapi integration is on top of both the starlette integration and the ASGI middleware, so the exception capturing is actually happening on a lower level during the ASGI app invocation, so even if the routes are loaded separately, it should technically work. https://github.com/getsentry/sentry-python/blob/bde87ff1a322e73a6aedb4fe6e9036c4d762fff1/sentry_sdk/integrations/asgi.py#L166-L168

But your code can help us repro this type of setup, so we'll keep you posted!

For the other SDK maintainers, I've updated this issue to an Enhancement since we seem to be missing support for some type of app setups.

[Arputikos]

Have the same problem. I am not able to install sentry in my project. No signal is coming, and doesn't matter if I init sentry before app or after the app.

[patrick91]

I had a similar issue and it got fixed by updating the sentry-sdk to latest 😊

[antonpirker]

Thanks @patrick91 for confirming that it works with the lastest SDK. I will close this issue.

If you come across this issue and have the same or similar problem, please create a new issue for it. Thanks!

[RemyNtshaykolo]

I was able to make it work with something like this

sentry_sdk.init(
    dsn="...",
    integrations=[FastApiIntegration(), StarletteIntegration()],
)


app = FastAPI(

)

app.openapi_version = "3.0.1"

# Configuration CORS
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Compression middleware
app.add_middleware(GZipMiddleware, minimum_size=1024_000)  # compress responses >= 1KB

@app.exception_handler(Exception)
async def catch_exceptions_middleware(request: Request, exc: Exception):
    sentry_sdk.capture_exception(Exception(exc))
    return JSONResponse(
        status_code=500,
        content={"detail": "Internal Server Error"},
        headers={
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Headers": "*",
            "Access-Control-Allow-Methods": "*",
        },
    )