Symbols not available for sentry native binary.

[eakoli]

Platform:

• [x] Android -> SDK 28, target 16
• [ ] Java -> If yes, which Java (and sourceCompatibility/targetCompatibility) version?
• [ ] Kotlin -> If yes, which Kotlin (and jvmTarget) version?
• [x] NDK -> NDK 17 gcc/gnustl_static
• [ ] React-Native -> If yes, which version?

IDE:

• [ ] Android Studio -> If yes, which version?
• [ ] IntelliJ -> If yes, which version?
• [ ] Other -> If yes, which one?

Build system:

• [x] Gradle -> 4.10.2
• [ ] Buck -> If yes, which version?
• [ ] Bazel -> If yes, which version?
• [ ] Other -> If yes, which one?

Android Gradle Plugin:

• [x] Yes -> 3.2.1
• [ ] No

Sentry Android Gradle Plugin:

• [x] Yes ->2.2.2
• [ ] No

Proguard/R8:

• [ ] Enabled
• [x] Disabled

sentry-android installed with:

• [ ] JCenter
• [ ] Bintray
• [x] Maven Central
• [ ] Manually

The version of sentry-android: 2.2.2

---

I have the following issue:

Having a crash that seems to be from sentry, but there are no symbols available to diagnose.

OS Version: Android 6.0.1 (MMB29M.G900VVRU2DQL1) Report Version: 104

Exception Type: Unknown (SIGABRT)

Application Specific Information: Abort

Thread 0 Crashed: 0 libc.so 0xb6c87ff8 tgkill 1 libc.so 0xb6c85c09 pthread_kill 2 libc.so 0xb6c6238f raise 3 libc.so 0xb6c5f60d + 3066426893 4 libc.so 0xb6c5d560 + 3066418528 5 libsentry.so 0xb2d883d5 + 3000533973 6 libsentry.so 0xb2d88473 + 3000534131 7 libsentry.so 0xb2d7df15 [inlined] + 3000491797 8 libsentry.so 0xb2d7df15 [inlined] + 3000491797 9 libsentry.so 0xb2d7df15 [inlined] + 3000491797 10 libsentry.so 0xb2d7df15 [inlined] + 3000491797 11 libsentry.so 0xb2d7df15 [inlined] + 3000491797 12 libsentry.so 0xb2d7df15 [inlined] + 3000491797 13 libsentry.so 0xb2d7df15 [inlined] + 3000491797 14 libsentry.so 0xb2d7df15 [inlined] + 3000491797 15 libsentry.so 0xb2d7df15 [inlined] + 3000491797 16 libsentry.so 0xb2d7df15 [inlined] + 3000491797 17 libsentry.so 0xb2d7df15 [inlined] + 3000491797 18 libsentry.so 0xb2d7df15 [inlined] + 3000491797 19 libsentry.so 0xb2d7df15 [inlined] + 3000491797 20 libsentry.so 0xb2d7df15 [inlined] + 3000491797

Steps to reproduce:

• Step

Actual result:

• Actual

Expected result:

• Result

[marandaneto]

@eakoli thanks for reporting this. could you fill-up the version of your tooling? also, do you have more info to share like the device brand & model? can you reproduce this easily? right now we are in the dark here. cc @Swatinem

[Swatinem]

@marandaneto you said once that all the libs are pre-built on android, right? so as part of the buildstep, we would have to upload the symbols to some central location. In case people are building it themselves, we would also have some kind of buildstep to upload symbols.

[marandaneto]

@Swatinem our Gradle plugin does it. https://docs.sentry.io/platforms/android/#gradle-configuration

they just need to enable uploadNativeSymbols.

@eakoli you've answered yes for Sentry Android Gradle Plugin, did you enable uploadNativeSymbols too? so it could upload symbols for your own lib and also Sentry lib if necessary.

[eakoli]

I have tried to use uploadNativeSymbols, but I cannot get it to work( never see any attempt to upload during the build), so we are using the sernry-cli tool to do so from cmake ( the same way we manage uploading for Mac/win/linux)

[eakoli]

The issue is happening in production only (never had the issue in any of our initial testing). Its occurring across numerous SM-XXX devices all on android 8.0, all armv7a

here is the sentry issue if that would help https://sentry.io/organizations/hubstaff/issues/1827115721/?project=5277412&query=is%3Aunresolved

[eakoli]

@marandaneto @Swatinem I have traced down the root cause of our crash.

If you look at the symbols exported from the libsentry.so included with sentry-android it not only exports all the sentry_xxx functions, but it also exports symbols for unwindstack and the c++ runtime. I believe the intention (Based on looking at the visibility settings in sentry-native) that only the sentry_xxx routines should be exported.

Our library is statically linked to gnustl, but its ending up using the stl symbols from lib sentry, which are not compatible.

It looks like android cmake setup always links the stl last, thus the symbols are resolved from lib sentry instead of the static lib.

I have worked around the issue temporarily (both the linking issue and the lack of debug symbols for libsenry.so) by adding sentry-native as a submodule any building it as part of our code base, preferring that version over the version bundled in sentry-core.

I limited the exported symbols using a version-script

-Wl,--version-script=sentry_export.map

sentry_export.map:

{
   global: sentry_*;
   local: *;
};

[marandaneto]

I have tried to use uploadNativeSymbols, but I cannot get it to work( never see any attempt to upload during the build), so we are using the sernry-cli tool to do so from cmake ( the same way we manage uploading for Mac/win/linux)

what is the issue? have you configured the sentry.properties file properly? https://docs.sentry.io/platforms/android/#gradle-integration

[marandaneto]

@marandaneto @Swatinem I have traced down the root cause of our crash.

If you look at the symbols exported from the libsentry.so included with sentry-android it not only exports all the sentry_xxx functions, but it also exports symbols for unwindstack and the c++ runtime. I believe the intention (Based on looking at the visibility settings in sentry-native) that only the sentry_xxx routines should be exported.

Our library is statically linked to gnustl, but its ending up using the stl symbols from lib sentry, which are not compatible.

It looks like android cmake setup always links the stl last, thus the symbols are resolved from lib sentry instead of the static lib.

I have worked around the issue temporarily (both the linking issue and the lack of debug symbols for libsenry.so) by adding sentry-native as a submodule any building it as part of our code base, preferring that version over the version bundled in sentry-core.

I limited the exported symbols using a version-script

-Wl,--version-script=sentry_export.map

sentry_export.map:

{
   global: sentry_*;
   local: *;
};

I see, glad that you found a workaround, although I'm not familiar with this type of setup, I'll let @Swatinem to take this over and let me know if we need to do any changes on the sentry-android cmake setup, thanks.

[marandaneto]

btw @eakoli could you let us know what exactly have you done so if anybody else runs into the same issue, we'd be able to support them as well? thanks a bunch.

[eakoli]

basically I just have private build of libsenry that has the exported symbols restricted so that it doesn't end up exporting the STL. As explained in the previous comment.

The work to get a private build is a bit of a pain, so it wouldn't suggest anyone having to go this route, best to fix sentry-native and then update sentry-android.

[marandaneto]

thanks @eakoli

[bruno-garcia]

We're uploading symbols on all new releases. Please upgrade the SDK.

[eakoli]

@bruno-garcia The core issue was not a problem with symbols not being uploaded (that was just a hinderance in figuring out the core issue) the main issue is the exported symbol setup that lib sentry uses. Which I don't see any linked PR/issue to address that core problem.

[bruno-garcia]

Maybe @Swatinem can help us out here

[marandaneto]

@Swatinem friendly reminder :)

[Swatinem]

There used to be a PR in the sentry-native repo about this: https://github.com/getsentry/sentry-native/pull/363 This pretty much stalled, but I can try to reopen it.

[marandaneto]

@Swatinem should we maybe move this issue to sentry-native repo then? Since the changes are in there.

[bruno-garcia]

Moved to sentry-native since changes need to happen there (see: https://github.com/getsentry/sentry-native/pull/363)

[romtsn]

@Swatinem can we do something to push the PR through and get it merged? How much work does it still need approximately? Otherwise we could just close it as not planned, since it's open for ~3 years now.

[supervacuus]

If I have a say in this, I would prefer this to stay open. Only recently, we got an issue that is connected to our large exported symbol surface, and this will be a recurring maintenance issue. So this needs to be fixed.

[supervacuus]

This has (just now) been released with 0.6.0. I'll close this and recommend creating new issues on top of 0.6.0 since this is a breaking change and the core context of this issue is resolved. Thank you all involved for your patience :-)