Sentry still sends events when before_send is configured to drop them

[tevansuk]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.9.10

Steps to Reproduce

We configure sentry during tests, because in the past we've had bugs/interactions between instrumenting APIs like sentry or new-relic that occur when the integrations are enabled, but not when they are not enabled. Running tests without the integrations enabled therefore gives a false positive "success".

We do this by configuring sentry with a dummy DSN and passing a before_send event that drops all events:

def _dont_send_events(event, hint):
    return None


SENTRY_DSN = env("SENTRY_DSN")
before_send = None
if env("SENTRY_DONT_SEND_EVENTS") or env("TEST_MODE"):
    before_send = _dont_send_events
    SENTRY_DSN = "http://fake@dummy/123"

if SENTRY_DSN:
    sentry_sdk.init(
        dsn=SENTRY_DSN,
        environment=env("APP_ENVIRONMENT"),
        release=env("APP_RELEASE"),
        integrations=[DjangoIntegration()],
        before_send=before_send,
    )

However, since updating to sentry 1.3.1 to sentry 1.9.10, Sentry is now trying to send data to this dummy DSN:

--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/logging/__init__.py", line 1088, in emit
    stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
  File "/usr/local/lib/python3.8/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/venv/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 67, in run
    return old_run_func(self, *a, **kw)
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/venv/lib/python3.8/site-packages/sentry_sdk/worker.py", line 128, in _target
    callback()
  File "/venv/lib/python3.8/site-packages/sentry_sdk/transport.py", line 470, in send_envelope_wrapper
    self._send_envelope(envelope)
  File "/venv/lib/python3.8/site-packages/sentry_sdk/transport.py", line 387, in _send_envelope
    self._send_request(
  File "/venv/lib/python3.8/site-packages/sentry_sdk/transport.py", line 230, in _send_request
    response = self._pool.request(
  File "/venv/lib/python3.8/site-packages/urllib3/request.py", line 78, in request
    return self.request_encode_body(
  File "/venv/lib/python3.8/site-packages/urllib3/request.py", line 170, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/venv/lib/python3.8/site-packages/urllib3/poolmanager.py", line 376, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 812, in urlopen
    log.warning(
  File "/usr/local/lib/python3.8/logging/__init__.py", line 1458, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/usr/local/lib/python3.8/logging/__init__.py", line 1589, in _log
    self.handle(record)
  File "/usr/local/lib/python3.8/logging/__init__.py", line 1599, in handle
    self.callHandlers(record)
  File "/venv/lib/python3.8/site-packages/sentry_sdk/integrations/logging.py", line 96, in sentry_patched_callhandlers
    return old_callhandlers(self, record)
Message: "Retrying (%r) after connection broken by '%r': %s"
Arguments: (Retry(total=2, connect=None, read=None, redirect=None, status=None), NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9508276820>: Failed to establish a new connection: [Errno -5] No address associated with hostname'), '/api/123/envelope/')

The exception is caused by a open pytest bug.

Why does sentry now try to send data to the server, and how can I stop it?

Minimal test case

import sentry_sdk, logging
def dont_send(event, hint):
    return None

sentry_sdk.init(dsn="http://fake@dummy/123", before_send=dont_send)
logging.error("oh noes")
exit()

Expected Result

Python 3.8.14 (default, Oct  5 2022, 16:02:43) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sentry_sdk, logging
>>> def dont_send(event, hint):
...     return None
... 
>>> sentry_sdk.init(dsn="http://fake@dummy/123", before_send=dont_send)
<sentry_sdk.hub._InitGuard object at 0x7f2891dd7c40>
>>> logging.error("oh noes")
ERROR:root:oh noes
>>> exit()
> $ 

Actual Result

Python 3.8.14 (default, Oct  5 2022, 16:02:43) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sentry_sdk, logging
>>> def dont_send(event, hint):
...     return None
... 
>>> sentry_sdk.init(dsn="http://fake@dummy/123", before_send=dont_send)
<sentry_sdk.hub._InitGuard object at 0x7f2891dd7c40>
>>> logging.error("oh noes")
ERROR:root:oh noes
>>> exit()
WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2891fb11f0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/123/envelope/
Sentry is attempting to send 1 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit
WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2891fb13d0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/123/envelope/
WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2891fb1880>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/123/envelope/

[sl0thentr0py]

@tevansuk if you don't want to send events, just set the dsn to None instead of doing this before-send stuff.

[github-actions[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

[tevansuk]

:+1: