iOS and Android crash rate doesn't match

[pierricklete]

OS:

• [ ] Windows
• [X] MacOS
• [ ] Linux

Platform:

• [X] iOS
• [] Android

SDK:

• [X] @sentry/react-native (>= 1.0.0)
• [ ] react-native-sentry (<= 0.43.2)

SDK version: 5.19.0

react-native version: 0.70.13

Are you using Expo?

• [ ] Yes
• [X] No

Are you using sentry.io or on-premise?

• [X] sentry.io (SaaS)
• [ ] on-premise

If you are using sentry.io, please post a link to your issue so we can take a look:

iOS unexpected crash rate

Android expected crash rate

Configuration:

(@sentry/react-native)

const detectSoftCrash = (event: Sentry.Event) => {
  if (event.tags && event.tags['caughtByErrorBoundary']) {
    const exceptions = event?.exception?.values;
    forEach(exceptions, (exceptionValue) => {
      if (exceptionValue) {
        exceptionValue.mechanism = {
          type: 'onerror',
          handled: false,
        };
      }
    });
    event.level = 'fatal';
  }
  return event;
};

Sentry.init({ 
  environment,
  dsn,
  beforeSend: detectSoftCrash,
 })

---

I have the following issue:

context: We want to measure number of errors caught by Sentry React error boundary, as a rate depending of number of sessions

According documentation, session with unhandled errors should be marked as crashed

Setting explicitly mechanism:{type: 'onerror', handled: false} to a Sentry error event results into:

• (expected) an unhandled event on both platforms
• (expected) a crashed session on Android
• (unexpected) an healthy session on iOS

Steps to reproduce: using beforeSend, set mechanism={type: 'onerror', handled: false} to exceptions attached to a Sentry event

Actual result:

• on Android release dashboard, the crash rate considers unhandled errors
• on iOS release dashboard, the crash rate does not consider unhandled errors (even though they appear in unhandled issues table)

Expected result:

• on Android release dashboard, the crash rate does consider unhandled errors
• on iOS release dashboard, the crash rate does consider unhandled errors

[krystofwoldrich]

Hello @pierricklete, thank you for the message,

I've checked the data, and the THIS RELEASE column doesn't change with the selected environment, as since your iOS release has large adoption across environments the crash session rate is lower.

This seems like a product issue, I'll move this to the sentry repository.

When looking across all envs do the values align for you?

[getsantry[bot]]

Assigning to @getsentry/support for routing ⏲️

[pierricklete]

@krystofwoldrich

When looking across all envs do the values align for you?

no they don't align and they are not consistent across platforms

for better readability, here are new releases, from same codebase, sending a same and single unhandled error

on Android error affects crashed rate

on iOS error affects errored rate and failure rate

[pierricklete]

@krystofwoldrich Is it possible to have 2 different issues and keep one to understand the inconsistency of health metrics between platforms?

[pierricklete]

@krystofwoldrich same result/inconsistency using hint parameter of captureException

    Sentry.captureException(new Error('my error'), { mechanism: { handled: false } });

test releases with single and same event Android release with expected crash rate

iOS release with unexpected 0 crash rate

[krystofwoldrich]

@pierricklete Thank you for the additional information, I've moved the issue back to the RN SDK repository.

I'll try to reproduce the issue and keep you updated.

[krystofwoldrich]

I've investigated the issue and confirm that iOS does not mark the session as crashed when handled: false is set in JS.