Sentry + Zappa Integration Is Causing The Application To Break

[Prithvirajkumar]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

sentry.python.django 1.32.0

Steps to Reproduce

The customer has the following settings:

# in settings.py
from .sentry import sentry_init
...
if ENVIRONMENT == 'development':
    sentry_init(env)


# in sentry.py
def sentry_init(env):
    sentry_dsn = env("SENTRY_DSN")
    if not sentry_dsn:
        print("Missing SENTRY_DSN")
        return

    try:
        from sentry_sdk.integrations.aws_lambda import AwsLambdaIntegration
        from sentry_sdk.integrations.boto3 import Boto3Integration
        from sentry_sdk.integrations.django import DjangoIntegration
    except ModuleNotFoundError as exc:
        print('Unable to find module', exc)
        return

    sentry_sdk.init(
        dsn=env("SENTRY_DSN"),
        environment=env("ENVIRONMENT"),
        integrations=[
            AwsLambdaIntegration(),
            Boto3Integration(),
            DjangoIntegration(),
        ],
        send_default_pii=False,
        event_scrubber=EventScrubber(denylist=denylist),
        # before_send=strip_sensitive_data,
        before_send_transaction=filter_transactions,
        traces_sampler=traces_sampler,
    )

Initially they had a setup without defining the integrations, but even with that they get the following error: Error: ModuleNotFoundError: No module named 'sentry_sdk.integrations.boto3'

They believe this is related to https://github.com/zappa/Zappa

They then identified that the zappa configuration was setup to exclude any boto3* and botocore libraries

Context: Zappa needs to exclude the boto3 and botocore libraries because they are included in the lambda and the lambda won't work if we package with them, thus they need to be added to the exclude in zappa_settings.json. The zappa library uses the shutil.ignore_patterns library, so it is about figuring out what pattern will exclude the AWS boto libraries and allow other libraries with boto in the name. Changing boto3* to boto3/* excludes the boto3 library but no the Sentry sentry_sdk.integrations.boto3 library because boto3 is a boto3.py

They resolved this and tried deploying outside of their sandbox environment and ran into the same issue again.

Expected Result

The integration should not break the application.

Actual Result

The integration breaks the application.

[stryker-sk]

To bypass this error boto3 and botocore libraries need to be excluded from the exclude section in the zappa_settings.json, which will allow Sentry to work as expected, however this results in endpoint errors when trying to access the application.

DataNotFoundError('Unable to load data for: endpoints')

[antonpirker]

@stryker-sk can you please add information on what version of Django and Zappa you use? And if you have any special settings for Django and/or Zappa?

[antonpirker]

Hey @stryker-sk, I created a demo application to reproduce this error. It is a Django application using Zappa to run in AWS Lambda. The demo application first reads the contents of a S3 bucket and then raises an (DivisionByZero) error.

The errors and performance data is sent to Sentry as expected:

The error you get DataNotFoundError('Unable to load data for: endpoints') is also not coming from the Sentry SDK but from Boto3. So I think the problem has something to do with your project setup and not with the Sentry SDK.

One thing you could do is set sentry_sdk.init(debug=True) and then have a look at the CloudWatch logs of the Lambda function to see what is going on (on the Sentry side)

[stryker-sk]

Yeah, this is a fundamental component of zappa that breaks because of how zappa has to package the application.

This happens due to the naming of the Sentry library. Zappa needs to exclude the boto libraries during build/packaging in order to work on the lambda. If the libraries are not excluded, then this error happens. This is documented in the zappa code.

[antonpirker]

So you managed to exclude the boto3 library itself but keep the sentry boto integration and everything works now as expected?

[stryker-sk]

If the boto library is excluded, you cannot deploy the application.

[antonpirker]

Ok, sorry for the misunderstanding. Then I can not reproduce your issue. Here is the .zip of my Zappa project that works as excepted. It sends errors and performance data to Sentry and it can also use the Boto3 library to read the contents of a S3 Bucket: https://drive.google.com/file/d/1NmBNZigstBFPryha4OiockZ-fITkEjgZ/view?usp=sharing

Can you post your requirements.txt content, maybe we find something in there that causes this issue.

Mine is here: https://github.com/getsentry/test-awslambda-zappa-django/blob/main/requirements.txt

[stryker-sk]

Yes, I have no problem getting sentry to integrate. This issue is that when sentry is integrated the app doesn't work. Are you able to have a functioning app?

[sentrivana]

Hey @stryker-sk, the test app that Anton linked is up and working and we're able to send requests to endpoints and get a response as well as have the app report errors and performance data to Sentry. Could you please help us out with more information about your setup, like your requirements.txt, or any custom Django or Zappa settings/middleware/etc?

[stryker-sk]

What commands are you using to deploy to AWS? I can run the application locally with sentry without an issue. The problem is when it gets deployed to AWS.

[antonpirker]

I just used zappa deploy dev and zappa update dev to deploy/update. My Zappa settings you can see here: https://github.com/getsentry/test-awslambda-zappa-django

[szokeasaurusrex]

@stryker-sk Are you still experiencing this issue?

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀