React Native Javascript not symbolicated self-hosted backend error

[ayush-leap]

OS:

• [ ] Windows
• [ ] Linux
• [x] MacOS

Platform:

• [ ] iOS
• [x] Android

SDK:

• [x] @sentry/react-native (>= 1.0.0)
• [ ] react-native-sentry (<= 0.43.2)

SDK version: 5.22.2

react-native version: 0.71.19

Are you using Expo?

• [ ] Yes
• [x] No

Are you using sentry.io or on-premise?

• [ ] sentry.io (SaaS)
• [x] on-premise

If you are using sentry.io, please post a link to your issue so we can take a look:

[Link to issue]

Configuration:

(@sentry/react-native)

Sentry.init({
      dsn: Config.REACT_APP_SENTRY_DSN,
      tracesSampleRate: 1,
      sampleRate: 1,
      environment: "development",
      release: `${DeviceInfo.getBundleId()}@${DeviceInfo.getVersion()}+${DeviceInfo.getBuildNumber()}`,
      dist: DeviceInfo.getBuildNumber(),
      attachStacktrace: true,
      enableAutoPerformanceTracing: true,
      enabled: true,
      debug: false,
      integrations: [
        new Sentry.ReactNativeTracing({
          routingInstrumentation,
          enableStallTracking: true,
          enableAppStartTracking: true,
          enableNativeFramesTracking: true
        })
      ]
});

---

I have the following issue:

We recently migrated from RN 0.69.9 to 0.71.19 and stack traces started showing incorrectly. Currently i'm reporting it only for Android. As per this comment, I tried using metro-symbolicate and in it's output it did mentioned the exact line it happened but Sentry doesn't show the same.

I also tried adding Sentry AGP which wasn't present earlier but it also didn't solve the issue for me.

I'm triggering error as below.

  const onSendButtonPress = () => {
    captureException(new Error("TEST: v0.33.3 Send Error #2"))
  };

Output from npx metro-symbolicate index.android.bundle.map < stack.txt

Error: TEST: v0.33.3 Send Error #2
  at onSendButtonPress(src/screens/CosmosHome/components/TxButtonGroup/index.tsx:87:onSendButtonPress)
  at handleOnPress(src/screens/CosmosHome/components/TxButton/index.tsx:41:handleOnPress)
  at _performTransitionSideEffects(node_modules/react-native/Libraries/Pressability/Pressability.js:758:_performTransitionSideEffects)
  at _receiveSignal(node_modules/react-native/Libraries/Pressability/Pressability.js:695:_receiveSignal)
  at onResponderRelease(node_modules/react-native/Libraries/Pressability/Pressability.js:525:responderEventHandlers.onResponderRelease)
  at apply(native)
  at invokeGuardedCallbackImpl(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:22:invokeGuardedCallbackImpl)
  at apply(native)
  at invokeGuardedCallback(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:40:invokeGuardedCallback)
  at apply(native)
  at invokeGuardedCallbackAndCatchFirstError(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:53:invokeGuardedCallbackAndCatchFirstError)
  at executeDispatch(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:73:executeDispatch)
  at executeDispatchesAndReleaseTopLevel(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1125:executeDispatchesAndReleaseTopLevel)
  at forEach(native)
  at forEachAccumulated(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:361:forEachAccumulated)
  at anonymous(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1160:batchedUpdates$argument_0)
  at batchedUpdatesImpl(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:8457:batchedUpdatesImpl)
  at batchedUpdates(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1106:batchedUpdates)
  at _receiveRootNodeIDEvent(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1137:_receiveRootNodeIDEvent)
  at receiveTouches(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1209:ReactNativePrivateInterface.RCTEventEmitter.register$argument_0.receiveTouches)
  at apply(native)
  at __callFunction(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:427:__callFunction)
  at anonymous(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:113:__guard$argument_0)
  at __guard(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:368:__guard)
  at callFunctionReturnFlushedQueue(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:112:callFunctionReturnFlushedQueue)% 

android/app/build.gradle

apply plugin: "com.android.application"
apply plugin: "com.facebook.react"
apply plugin: "io.fabric"
apply plugin: "com.google.gms.google-services"
apply plugin: "io.sentry.android.gradle"

import com.android.build.OutputFile
...

/**
 * This is the configuration block to customize your React Native Android app.
 * By default you don't need to apply any configuration, just uncomment the lines you need.
 */
react {
    /* Folders */
    //   The root of your project, i.e. where "package.json" lives. Default is '..'
    // root = file("../")
    //   The folder where the react-native NPM package is. Default is ../node_modules/react-native
    // reactNativeDir = file("../node_modules/react-native")
    //   The folder where the react-native Codegen package is. Default is ../node_modules/react-native-codegen
    // codegenDir = file("../node_modules/react-native-codegen")
    //   The cli.js file which is the React Native CLI entrypoint. Default is ../node_modules/react-native/cli.js
    // cliFile = file("../node_modules/react-native/cli.js")
 
    /* Variants */
    //   The list of variants to that are debuggable. For those we're going to
    //   skip the bundling of the JS bundle and the assets. By default is just 'debug'.
    //   If you add flavors like lite, prod, etc. you'll have to list your debuggableVariants.
    debuggableVariants = ["leapdebug", "seidebug"]
 
    /* Bundling */
    //   A list containing the node command and its flags. Default is just 'node'.
    nodeExecutableAndArgs = ["node", "--max-old-space-size=8192"]
    //
    //   The command to run when bundling. By default is 'bundle'
    // bundleCommand = "ram-bundle"
    //
    //   The path to the CLI configuration file. Default is empty.
    // bundleConfig = file(../rn-cli.config.js)
    //
    //   The name of the generated asset file containing your JS bundle
    // bundleAssetName = "MyApplication.android.bundle"
    //
    //   The entry file for bundle generation. Default is 'index.android.js' or 'index.js'
    // entryFile = file("../js/MyApplication.android.js")
    //
    //   A list of extra flags to pass to the 'bundle' commands.
    //   See https://github.com/react-native-community/cli/blob/main/docs/commands.md#bundle
    // extraPackagerArgs = []
 
    /* Hermes Commands */
    //   The hermes compiler command to run. By default it is 'hermesc'
    // hermesCommand = "$rootDir/my-custom-hermesc/bin/hermesc"
    //
    //   The list of flags to pass to the Hermes compiler. By default is "-O", "-output-source-map"
    // hermesFlags = ["-O", "-output-source-map"]
}

// leap configuration for sentry AGP - https://docs.sentry.io/platforms/react-native/manual-setup/manual-setup/#enable-sentry-agp
sentry {
    // fixes: An organization ID or slug is required (provide with --org)
    org = "leapwallet"

    // fixes: A project ID or slug is required (provide with --project)
    projectName = "leap-cosmos-mobile"

    // List the build flavors that should be ignored (e.g. "production").
    // https://docs.sentry.io/platforms/android/configuration/gradle/#variant-filtering
    ignoredFlavors = ["sei", "dev"]

    // Enables or disables the automatic configuration of Native Symbols
    // for Sentry. This executes sentry-cli automatically so
    // you don't need to do it manually.
    // Default is disabled.
    uploadNativeSymbols = true

    // Enables or disables the automatic upload of the app's native source code to Sentry.
    // This executes sentry-cli with the --include-sources param automatically so
    // you don't need to do it manually.
    // This option has an effect only when [uploadNativeSymbols] is enabled.
    // Default is disabled.
    includeNativeSources = true

    // `@sentry/react-native` ships with compatible `sentry-android`
    // This option would install the latest version that ships with the SDK or SAGP (Sentry Android Gradle Plugin)
    // which might be incompatible with the React Native SDK
    // Enable auto-installation of Sentry components (sentry-android SDK and okhttp, timber and fragment integrations).
    // Default is enabled.
    autoInstallation {
      enabled = false
    }

    // Generates a JVM (Java, Kotlin, etc.) source bundle and uploads your source code to Sentry.
    // This enables source context, allowing you to see your source
    // code as part of your stack traces in Sentry.
    includeSourceContext = true
}

// compass configuration for sentry AGP - https://docs.sentry.io/platforms/react-native/manual-setup/manual-setup/#enable-sentry-agp
sentry {
    // fixes: An organization ID or slug is required (provide with --org)
    org = "leapwallet"

    // fixes: A project ID or slug is required (provide with --project)
    projectName = "compass-mobile"

    // List the build flavors that should be ignored (e.g. "production").
    // https://docs.sentry.io/platforms/android/configuration/gradle/#variant-filtering
    ignoredFlavors = ["leap", "dev"]

    // Enables or disables the automatic configuration of Native Symbols
    // for Sentry. This executes sentry-cli automatically so
    // you don't need to do it manually.
    // Default is disabled.
    uploadNativeSymbols = true

    // Enables or disables the automatic upload of the app's native source code to Sentry.
    // This executes sentry-cli with the --include-sources param automatically so
    // you don't need to do it manually.
    // This option has an effect only when [uploadNativeSymbols] is enabled.
    // Default is disabled.
    includeNativeSources = true

    // `@sentry/react-native` ships with compatible `sentry-android`
    // This option would install the latest version that ships with the SDK or SAGP (Sentry Android Gradle Plugin)
    // which might be incompatible with the React Native SDK
    // Enable auto-installation of Sentry components (sentry-android SDK and okhttp, timber and fragment integrations).
    // Default is enabled.
    autoInstallation {
      enabled = false
    }

    // Generates a JVM (Java, Kotlin, etc.) source bundle and uploads your source code to Sentry.
    // This enables source context, allowing you to see your source
    // code as part of your stack traces in Sentry.
    includeSourceContext = true
}

project.ext.sentryCli = [
    logLevel: "debug",
    flavorAware: true
]

apply from: new File(["node", "--print", "require.resolve('@sentry/react-native/package.json')"].execute().text.trim(), "../sentry.gradle")
...

/**
 * Set this to true to Run Proguard on Release builds to minify the Java bytecode.
 */
def enableProguardInReleaseBuilds = false


android {
    ...
    flavorDimensions 'default'

    productFlavors {
        leap {
            applicationId "io.leapwallet.cosmos"
            minSdkVersion rootProject.ext.minSdkVersion
            targetSdkVersion rootProject.ext.targetSdkVersion
            resValue "string", "build_config_package", "io.leapwallet.cosmos"
            versionCode 359
            versionName "0.0.3"
        }
        dev {
            applicationId "io.leapwallet.cosmos.dev"
            minSdkVersion rootProject.ext.minSdkVersion
            targetSdkVersion rootProject.ext.targetSdkVersion
            resValue "string", "build_config_package", "io.leapwallet.cosmos"
            versionCode 359
            versionName "0.0.3"
        }
        sei {
            applicationId "io.leapwallet.compass"
            minSdkVersion rootProject.ext.minSdkVersion
            targetSdkVersion rootProject.ext.targetSdkVersion
            resValue "string", "build_config_package", "io.leapwallet.cosmos"
            versionCode 359
            versionName "0.0.3"
        }
    }

}

`build.gradle`

    dependencies {
        // RN Upgrade helper mentions 7.3.1 but expo modules docs says to update it to 7.4.1.
        // refer: https://docs.expo.dev/bare/upgrade/?fromSdk=46&toSdk=48#androidbuildgradle
        classpath("com.android.tools.build:gradle:7.4.1") 
        classpath("com.facebook.react:react-native-gradle-plugin")
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
        classpath("com.google.gms:google-services:4.3.15")
        classpath('io.fabric.tools:gradle:1.28.1')
        classpath("org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlinVersion")
        // enable sentry AGP - https://docs.sentry.io/platforms/react-native/manual-setup/manual-setup/#enable-sentry-agp
        classpath("io.sentry:sentry-android-gradle-plugin:4.2.0")
    }

metro.config.js

/**
 * Metro configuration for React Native
 * https://github.com/facebook/react-native
 *
 * @format
 */

const { getDefaultConfig } = require("metro-config");
const { withSentryConfig } = require("@sentry/react-native/metro");

module.exports = (async () => {
  const {
    resolver: { sourceExts, assetExts }
  } = await getDefaultConfig();

  const config = {
    transformer: {
      experimentalImportSupport: false,
      inlineRequires: true,
      babelTransformerPath: require.resolve("react-native-svg-transformer")
    },
    resolver: {
      assetExts: assetExts.filter((ext) => ext !== "svg"),
      sourceExts: [...sourceExts, "svg", "jsx", "js", "ts", "tsx", "cjs", "mjs"],
      extraNodeModules: require("node-libs-react-native")
    }
  };

  // Ensures all Sentry configuration is added to your Metro config
  // Sentry Metro Serializer which allows you to automatically generate Debug IDs for your applications' bundles. This is crucial for making source maps work properly with Sentry
  // withSentryConfig will remove any existing customSerializer. Guide for advanced setups can be found here - https://docs.sentry.io/platforms/react-native/manual-setup/metro/#wrap-your-custom-serializer.
  // Sentry Metro Serializer can't add Debug ID to the Hermes Composed Source Maps.
  // refer - https://docs.sentry.io/platforms/react-native/manual-setup/metro/
  return withSentryConfig(config);
})();

Steps to reproduce:

• Step 1
• Step 2

Actual result:

Expected result:

The stack track should point to the exact line & file the error triggered from.

[ayush-leap]

I tried upgrading to v5.30.0 as well, facing same issue.

[ayush-leap]

Same issue happening on iOS as well. Tested iOS with RN 0.71.19 + Sentry 5.30.0

Artifact Bundles

Sentry Stack Trace

`npx metro-symbolicate main.jsbundle.map

Error: ios TEST: v0.33.3 vote Error 0.0.5
  at onVoteButtonPress(src/screens/CosmosHome/components/TxButtonGroup/index.tsx:113:onVoteButtonPress)
  at handleOnPress(src/screens/CosmosHome/components/TxButton/index.tsx:41:handleOnPress)
  at _performTransitionSideEffects(node_modules/react-native/Libraries/Pressability/Pressability.js:758:_performTransitionSideEffects)
  at _receiveSignal(node_modules/react-native/Libraries/Pressability/Pressability.js:695:_receiveSignal)
  at onResponderRelease(node_modules/react-native/Libraries/Pressability/Pressability.js:525:responderEventHandlers.onResponderRelease)
  at apply(native)
  at invokeGuardedCallbackImpl(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:22:invokeGuardedCallbackImpl)
  at apply(native)
  at invokeGuardedCallback(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:40:invokeGuardedCallback)
  at apply(native)
  at invokeGuardedCallbackAndCatchFirstError(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:53:invokeGuardedCallbackAndCatchFirstError)
  at executeDispatch(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:73:executeDispatch)
  at executeDispatchesAndReleaseTopLevel(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1125:executeDispatchesAndReleaseTopLevel)
  at forEach(native)
  at forEachAccumulated(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:361:forEachAccumulated)
  at anonymous(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1160:batchedUpdates$argument_0)
  at batchedUpdatesImpl(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:8457:batchedUpdatesImpl)
  at batchedUpdates(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1106:batchedUpdates)
  at _receiveRootNodeIDEvent(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1137:_receiveRootNodeIDEvent)
  at receiveTouches(node_modules/react-native/Libraries/Renderer/implementations/ReactNativeRenderer-prod.js:1209:ReactNativePrivateInterface.RCTEventEmitter.register$argument_0.receiveTouches)
  at apply(native)
  at __callFunction(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:427:__callFunction)
  at anonymous(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:113:__guard$argument_0)
  at __guard(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:368:__guard)
  at callFunctionReturnFlushedQueue(node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:112:callFunctionReturnFlushedQueue)%

Bundle React Native code and images

export NODE_OPTIONS="--max-old-space-size=8192"
export SENTRY_PROPERTIES=sentry-leap-release.properties

set -e

WITH_ENVIRONMENT="../node_modules/react-native/scripts/xcode/with-environment.sh"
REACT_NATIVE_XCODE="../node_modules/react-native/scripts/react-native-xcode.sh"
SENTRY_XCODE="../node_modules/@sentry/react-native/scripts/sentry-xcode.sh"
BUNDLE_REACT_NATIVE="/bin/sh $SENTRY_XCODE $REACT_NATIVE_XCODE"

/bin/sh -c "$WITH_ENVIRONMENT \"$BUNDLE_REACT_NATIVE\""

# send runtime JavaScript dependencies in events for React Native apps
/bin/sh ../node_modules/@sentry/react-native/scripts/collect-modules.sh

Upload Debug Symbols to Sentry

export NODE_OPTIONS=--max-old-space-size=8192
export SENTRY_PROPERTIES=sentry-leap-release.properties

/bin/sh ../node_modules/@sentry/react-native/scripts/sentry-xcode-debug-files.sh

[kahest]

@ayush-leap thank you for the detailed report, we'll investigate and follow up here

[krystofwoldrich]

Hi @ayush-leap, thank you for the message,

have you also upgraded the Sentry React Native SDK when changing from [email protected] to [email protected]? Were you using the Sentry Metro Plugin @sentry/react-native/metro which generates Debug IDs?

What version of Sentry Self hosted are you using? Have you check the backend logs, is are there any symbolication errors?

[ayush-leap]

have you also upgraded the Sentry React Native SDK when changing from [email protected] to [email protected]?

@krystofwoldrich no, during the RN upgrade we didn't upgrade Sentry as we were already on 5.22.2. after we pushed upgrade live, saw stack traces issue & only then I tried to upgrade to 5.30.0 to check if it resolves the issue but it didn't.

Were you using the Sentry Metro Plugin @sentry/react-native/metro which generates Debug IDs?

yes, we were using sentry's metro plugin previously as well. you can refer the implementation in PR description, it's unchanged.

What version of Sentry Self hosted are you using? Have you check the backend logs, is are there any symbolication errors?

we're using sentry's 24.1.0 at backend. yes, we did see below error log at backend, but team is unsure what it means or ways to resolve it. is this a backend issue? if yes, any tips on resolving this?

backend logs

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='symbolicator', port=3021): Max retries exceeded with url: /symbolicate-js?timeout=5&scope=5 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3364249d20>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sentry/tasks/symbolication.py", line 249, in _do_symbolicate_event
    symbolicated_data = symbolication_function(symbolicator, data)
  File "/usr/local/lib/python3.10/site-packages/sentry/lang/javascript/processing.py", line 240, in process_js_stacktraces
    response = symbolicator.process_js(
  File "/usr/local/lib/python3.10/site-packages/sentry/lang/native/symbolicator.py", line 219, in process_js
    return self._process("symbolicate_js_stacktraces", "symbolicate-js", json=json)
  File "/usr/local/lib/python3.10/site-packages/sentry/lang/native/symbolicator.py", line 105, in _process
    json_response = session.create_task(path, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sentry/lang/native/symbolicator.py", line 359, in create_task
    return self._request(method="post", path=path, params=params, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sentry/lang/native/symbolicator.py", line 293, in _request
    response = self.session.request(method, url, timeout=self.timeout + 1, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sentry/net/http.py", line 189, in request
    response = _Session.request(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)

[krystofwoldrich]

@ayush-leap Thank you for the details, it looks like the symbolication fails on the backend.

I'll forward this to my colleagues, who can take a look at the backend error.

[ayush-leap]

I'll forward this to my colleagues, who can take a look at the backend error.

@krystofwoldrich appreciate it, waiting for the response.

[hubertdeng123]

Hi there @ayush-leap . It looks like your web instance is not able to connect to symbolicator. Can you check your symbolicator logs to see if anything out of the ordinary is going on?

docker compose logs symbolicator

Could you also paste the result of

docker compose ps

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

[krystofwoldrich]

Closing this due to inactivity. If this is still an issue feel free to comment here or open a new issue if the context changed.