Java Source Context

[fprochazka]

Summary

I would like to be able to see sources in stacktraces of java exceptions, the same way they're visible for javascript.

Motivation

This would improve my ability to see the problem in the context of my application.

Additional Context

Sentry java client, should be able to access classpath of the running application and build process should be able to provide versions of installed libraries and built artefacts. Combining these two informations, Sentry should be able to render source code of Java. This would allow you to render on at least some frames of the stacktrace (there will always be generated/proxy frames, that will have no "visible" source code available).

By uploading correct artifact (with sources) to a release, I should be able fill in the main part - source code of my own application.

And it would be ideal, if fetching "public" maven jars with sources could be a native part of Sentry, since these libraries take up a massive amount of disk storage, and if they would duplicate on each release, this would be unfeasible. Having a shared cache between releases would be ideal.

Refs

• https://github.com/getsentry/sentry/issues/10435

[yurik94]

Still not possible?

[arvkonstantin]

Still not possible?

[romtsn]

Still not possible?

[mitsuhiko]

So I tried doing this a few years ago and mainly ran into complications with the tooling and introspection. For source context to work we would like to ideally piggyback on the source bundle system we use for a few other platforms. For this to work we need to do the following things:

• send up the ID of the source bundle with the event is sent
• get the file name of the file that declares a java class
• look up the source by file name in the source bundle

There are a handful of restrictions unfortunately. The biggest one is that the file name does not actually exist in the stack trace. This is not an unsolvable problem as we could derive made up file names based on the class path (replace . with / and add .java to the end). However I was unable to actually create the source bundle in a good way as there (at the time) did not appear to be a standardized way to hook into the .jar creation process to extract the source files.

There were also too many different build systems (gradle, maven etc.) to make this work and a lot of code uses published maven archives and it was unclear how we get access to those sources.

So maybe a new approach could be found where we leverage the gradle plugin for Android as a first step and start creating source bundles there. For the build ID to give the bundle we could use a similar system as we already use for the proguard files. Then we have a starting point and can see if there is a way to expand it from there.

[fprochazka]

The build system is aware of the full dependency tree for a given JAR artifact and a custom maven/gradle plugin should be able to dump the info into some resource. If the format is simple enough, I believe the community would have no problem providing implementations for less common build systems.

Once you have that info, you can attach it to the event, or publish it as a release artifact (better).

Creating a hack that would derive filenames from class paths is IMHO a bad idea, as JVM doesn't really care much about filenames, it cares about classpath/modulepath. The final classpath/modulepath can be also derived from build system and included in the artefact describing the dependencies. Once you know the installed dependencies and the final classpath, you can then fetch the dependencies and lookup the correct file using the classpath.

Most artifacts can be found on public maven registry, but some will be privately published on company's internal package registry, which would make fetching the sourcecode a bit harder.

[maciejwalkowiak]

Most artifacts can be found on public maven registry, but some will be privately published on company's internal package registry, which would make fetching the sourcecode a bit harder.

I think it's fair to assume that if a company uses internal package registry, to get source context support they must publish there sources jars as well.

[adinauer]

We have released Java Source context for Android, Backends, Desktop, ... now. Below you can find some more details on how this works under the hood, limitations, etc.

Docs

• Android
• Java

Known limitations

• We only support Gradle and Maven at the moment. You may invoke sentry-cli manually to enjoy Source Context with other build tools. If you're interested, please take a look at the "Implementation Details" in this comment
• Multiple files with same name but different extension will lead to undefined behaviour
  • e.g. MainActivity.java and MainActivity.kt will both be renamed to MainActivity.jvm
• Package declaration and file tree must match for source lookup to work
  • e.g. a class io.sentry.sample.MainActivity.java has to be stored in io/sentry/sample
• Kotlin files may contain multiple classes but this feature may be broken by code obfuscation tools like ProGuard or R8
• For AGP < 7.4 we don't add generated sources yet
• For the Maven plugin you have to manually download sentry-cli and point the plugin to it

Minimum required versions

• Java SDK: https://github.com/getsentry/sentry-java/releases/tag/6.22.0
• sentry-android-gradle-plugin: https://github.com/getsentry/sentry-android-gradle-plugin/releases/tag/3.10.0
• sentry-maven-plugin: https://github.com/getsentry/sentry-maven-plugin/releases/tag/0.0.2
• sentry-cli: https://github.com/getsentry/sentry-cli/releases/tag/2.17.3 (sentry-android-gradle-plugin should automatically download this for you)
• sentry: https://github.com/getsentry/sentry/releases/tag/23.5.0 (in case you're self hosting)
• relay: https://github.com/getsentry/relay/releases/tag/23.4.0 (in case you're self hosting)

Follow Up Tasks

• [x] Release as experimental
• [x] Docs
• [x] Maven Plugin
• [x] Gradle Plugin for Backend builds
• [ ] Speed up plugin e.g. by symlinking files instead of copying; needs test on how slow this actually is for larger projects - if you have feedback, please leave a comment 🙏

What it looks like in Sentry UI

Screenshot


Implementation Details

Implementation Details

### During build time

In Build System

• [x] PR: https://github.com/getsentry/sentry-android-gradle-plugin/pull/472
• [x] In build system (gradle, SAGP for starters) we have to hook into the build and gain access to the sources
• [x] Copy sources into a build/reports/sentry-source-bundle/<VARIANT>/intermediates/sentry/source-to-bundle dir
• [x] Call sentry-cli difutil bundle-jvm <PATH> to bundle sources
• [x] Call sentry-cli upload-dif --type=jvm <SOURCE_BUNDLE.ZIP> to upload source bundle
• [x] Embed bundle ID into AndroidManifest so it can be read at runtime

sentry-cli changes

• [x] New command (e.g. sentry-cli difutil bundle-jvm <PATH>) Turn source into a bundle and generate debug ID (https://github.com/getsentry/sentry-cli/pull/1551)
• [x] Support for a new debug file type jvm

Relay changes

• [x] Support for a new debug file type jvm (https://github.com/getsentry/relay/pull/2002)

Changes in Sentry

• [x] New processor for adding context_line, pre_context and post_context to stack frames (https://github.com/getsentry/sentry/pull/46497)
  • This delegates to the existing JavaStacktraceProcessor which uses ProGuard / R8 mapping files to deobfuscate code before looking up source context

During runtime

• [x] PR: https://github.com/getsentry/sentry-java/pull/2663
• [x] Read all available bundle IDs from AndroidManifest
• [x] Send bundle IDs with events

For matching running code to source

• While bundling sources we replace the real file extension with a fake one (.jvm)
• We look at module of the stack frame
• We look at abs_path (sent as filename by Java SDK)
  • If it looks valid we extract the package from module and convert it to a path (. -> /) and append abs_path to obtain the source path to look up in the bundle, replacing the file extension with .jvm. So module=io.sentry.samples.spring.boot.AnotherService$InnerClassOfAnotherService$InnerClassLvl2OfAnotherService and abs_path=PersonController.kt becomes io/sentry/samples/spring/boot/PersonController.jvm
  • If it looks obfuscated (contains $) we ignore abs_path and use only module for obtaining the path of the source file by stripping possible inner classes ($...) from the class name and appending .jvm. So io.sentry.samples.spring.boot.AnotherService$InnerClassOfAnotherService$InnerClassLvl2OfAnotherService becomes io/sentry/samples/spring/boot/AnotherService.jvm

Other TODOs

• [x] ~~Might need to update manifest structure of source bundles to support dotted notation instead of paths~~
• [x] ~~Probably won't need symbolicator but there's no way to just look up source code yet, may need to change that~~ Changes are only done in python part of Sentry, see above
• [x] We ~~may~~ need a new debug file type: jvm

What do jvm source bundles look like?

Entries are source files in a tree structure with file extension replaced by .jvm. In combination with code obfuscation tools like ProGuard or R8 we sometimes encounter filenames for stack frames that are obfuscated / made up. This means we can't know the file extension of the source file. In order not to have to guess the file extension we decided to use a fake extension (.jvm) instead. This means MainActivity.java is renamed to MainActivity.jvm and SampleActivity.kt is renamed to SampleActicity.jvm.

Example of contents:

• io/sentry/sample/MainActivity.java
  • stored as io/sentry/sample/MainActivity.jvm
  • shows up in stack frame as io.sentry.sample.MainActivity
• io/sentry/sample/SampleActivity.kt
  • stored as io/sentry/sample/SampleActivity.jvm
  • shows up in stack frame as io.sentry.sample.SampleActivity

Things to test

• [x] Android
• [x] Android with ProGuard turned on
• [ ] Bytecode manipulation on Android
• [ ] Java Agent manipulation on backend
• [ ] Generated code
• [x] Kotlin code

[adinauer]

We've just released Source Context support for Android. Please give it a try: https://github.com/getsentry/sentry-android-gradle-plugin/releases/tag/3.7.0 (which uses https://github.com/getsentry/sentry-java/releases/tag/6.19.0)

Any feedback is welcome 🙏

We're still working on adding Source Context for Backend etc. - stay tuned.

[beheh]

I'm not quite sure where to ask this, but I thought here was more appropriate than the sentry-android-gradle-plugin PR and because you specifically asked for feedback:

Is there a path to source code uploading for Android with isMinifyEnabled set to false? For example, in a small app with have disabled minification for better stack traces and reproducibility. However it seems right now minification is a requirement for uploading (as per getsentry/sentry-android-gradle-plugin#86).

Or maybe this is more of a hosted Sentry question, where we're used to have source code available for non-processed files in e.g. Python and/or direct links to GitHub. It would be nice if we were able to have a similar experience in the issue interface to see source code directly in Sentry for context.

[romtsn]

I'm not quite sure where to ask this, but I thought here was more appropriate than the sentry-android-gradle-plugin PR and because you specifically asked for feedback:

Is there a path to source code uploading for Android with isMinifyEnabled set to false? For example, in a small app with have disabled minification for better stack traces and reproducibility. However it seems right now minification is a requirement for uploading (as per getsentry/sentry-android-gradle-plugin#86).

Or maybe this is more of a hosted Sentry question, where we're used to have source code available for non-processed files in e.g. Python and/or direct links to GitHub. It would be nice if we were able to have a similar experience in the issue interface to see source code directly in Sentry for context.

The PR linked is rather about uploading proguard mappings, but not the source context. For source context we just hook into the release build type. So if you're running assembleRelease and have the includeSourceContext flag enabled, you should be good.

As for self-hosted - yes you'd have to update to the latest version, as there were some necessary changes on the stacktrace processing.

[beheh]

I see now, I spoke too soon. With includeSourceContext explicitly set it all works! The line above ("If you aren't using ProGuard (yet), you'll have to supply some properties.") confused me because it made it sound like ProGuard is a requirement. Thanks!

[adinauer]

I've added another sentence to clarify ProGuard isn't a requirement for using Source Context.

[adinauer]

We have now released Java Source Context support for Backend, Desktop, etc. via Gradle and Maven as well. Please take a look at https://github.com/getsentry/sentry-java/issues/633#issuecomment-1465599120 for more details.

Docs are coming soon.

[adinauer]

Docs shave now been merged and can be found here:

• Android
• Java

Closing this now. We can do improvements in separate issues.

[androidacy-user]

Docs shave now been merged and can be found here:

• Android
• Java

Closing this now. We can do improvements in separate issues.

Um is there a reason we can't specify the URL for self hosted in Gradle?

EDIT: okay so docs don't bother to mention that the usual sentry.properties is still respected for this, instead suggesting you have to specify the org/project/auth token in build.gradle.kts

[adinauer]

@androidacy-user thanks for bringing this up, we'll improve docs. I've created https://github.com/getsentry/sentry-java/issues/2854 to track.