sentry-symfony icon indicating copy to clipboard operation
sentry-symfony copied to clipboard

Sentry timeouts don't work on our server

Open xevolic opened this issue 1 year ago • 7 comments

How do you use Sentry?

Self-hosted / on-premises

SDK version

4.5.0

Steps to reproduce

We have an application, which we develop in Docker environment.

We added three options to Sentry configuration in sentry.yaml file:

    options:
        attach_stacktrace: true
        send_attempts: '%env(int:SENTRY_RESEND_ATTEMPTS)%'
        http_connect_timeout: '%env(int:SENTRY_HTTP_CONNECT_TIMEOUT)%'
        http_timeout: '%env(int:SENTRY_HTTP_TIMEOUT)%'
        environment: '%kernel.environment%'
        error_types: 'E_ALL & ~E_NOTICE & ~E_DEPRECATED'
        release: '%kernel.environment%'
        integrations:
            - 'Sentry\Integration\IgnoreErrorsIntegration'

Our values are:

SENTRY_RESEND_ATTEMPTS=0
SENTRY_HTTP_CONNECT_TIMEOUT=1
SENTRY_HTTP_TIMEOUT=2

We tested these timeouts in Docker environment and all response times from our backend application were correct.

But when we deployed this application to our server they seems to be ignored.

We tested Sentry timeout with some page with sleep(10); to get response after 10 sec.

This approach worked in Docker, but does not work on the server: we need to wait these 10 seconds for Sentry, but it should be quit after 2 seconds (timeout limit reached).

Can you help us in finding what we are doing wrong?

Expected result

Sentry timeout should work as expected.

Actual result

We need to wait for e..g 10 seconds for page, which simulates Sentry.

xevolic avatar Nov 10 '23 11:11 xevolic

The defaults are applied by the underlying PHP SDK here https://github.com/getsentry/sentry-php/blob/3.x/src/Options.php#L25-L35. Hence I'm wondering why you see a 10 second timeout, it almost feels like that the env vars you set on production somehow set these options to null.

Another possibility could be that you're using an HTTP Client we do not "support", as in we don't apply the options here https://github.com/getsentry/sentry-php/blob/3.x/src/HttpClient/HttpClientFactory.php#L112-L164

cleptric avatar Nov 10 '23 12:11 cleptric

@cleptric thanks for your reply

I forgot to mention that our application uses Symfony framework and uses following configuration file (sentry.yaml):

sentry:
    dsn: '%env(SENTRY_DSN)%'

#    If you are using Monolog, you also need these additional configuration and services to log the errors correctly:
#    https://docs.sentry.io/platforms/php/guides/symfony/#monolog-integration
    register_error_listener: false

    options:
        attach_stacktrace: true
        send_attempts: '%env(int:SENTRY_RESEND_ATTEMPTS)%'
        http_connect_timeout: '%env(int:SENTRY_HTTP_CONNECT_TIMEOUT)%'
        http_timeout: '%env(int:SENTRY_HTTP_TIMEOUT)%'
        environment: '%kernel.environment%'
        error_types: 'E_ALL & ~E_NOTICE & ~E_DEPRECATED'
        release: '%kernel.environment%'
        integrations:
            - 'Sentry\Integration\IgnoreErrorsIntegration'

We try to simulate 10-second delay of Sentry application to check if timeout values are working as expected.

Our application connects to SMTP server and if it rejects our request, an error is sent to Sentry. In that case, with high traffic, we killed Sentry some time ago and want to prevent it from happening again.

We can increase 10-second delay to 15 or 30 seconds... we just chose 10 at the moment.

xevolic avatar Nov 10 '23 17:11 xevolic

The Symfony SDK uses the PHP SDK under the hood https://github.com/getsentry/sentry-symfony/blob/master/composer.json#L31, so I just wanted to point out where the default values are coming from. I assume something is wrong with your production server, as the config values are correctly applied in my tests.

If you are generally concerned about the overhead the PHP SDK is introducing to your request times, I recommend running a local Relay next to your application https://docs.sentry.io/platforms/php/guides/symfony/performance/#improve-response-time.

This decreases the roundtrip time, and even if your on-premise instance is not accepting any events, the local Relay will still respond with a 200 status code.

cleptric avatar Nov 10 '23 20:11 cleptric

We made a workaround by replacing default timeout values in vendor/sentry/sentry/src/Options.php (with sed), because setting timeouts in sentry.yaml didn't work for us.

xevolic avatar Nov 15 '23 16:11 xevolic

You mentioned it was working in Docker but not in your production environment. Did you try setting the raw values in your sentry.yaml instead of using environment variables?

cleptric avatar Nov 15 '23 16:11 cleptric

Hi there.

Yes, I tried setting raw values in sentry.yaml, but they didn't work for me (they were not been applied on the server).

I'm not sure if it could help you, but when I was debugging the issue:

  • in file vendor/sentry/sentry/src/Client.php:
    • I got locally in Docker:
captureEvent() :: 2023-11-13T20:35:20.235179 :: start
captureEvent() :: 2023-11-13T20:35:22.240692 :: Operation timed out after 2001 milliseconds with 0 bytes received for "https://beta.somewebapp.pl/api//store/".
captureEvent() :: 2023-11-13T20:35:22.240692 :: Symfony\Component\HttpClient\Exception\TransportException
  • and on the server:
captureEvent() :: 2023-11-13T21:14:14.218842 :: start
captureEvent() :: 2023-11-13T21:14:16.222341 :: The promise was rejected
captureEvent() :: 2023-11-13T21:14:16.222341 :: GuzzleHttp\Promise\RejectionException

xevolic avatar Nov 17 '23 19:11 xevolic

Please try to update the SDK composer update -W sentry/sentry-symfony as you're running a quite old version of the SDK.

cleptric avatar Nov 20 '23 05:11 cleptric

Closing due to inactivity.

cleptric avatar Apr 08 '24 08:04 cleptric