Support multi-module Android projects when sending Source Context

[wzieba]

Problem Statement

When sending Source Context via Sentry Gradle Plugin, I've noticed that the source is taken only from the main application module. This is problematic for multi-module setups, which are fairly common.

I've prepared a minimal reproduction project of what I have in mind.

Reproduction

An Android application with x app module and y its dependency: https://github.com/wzieba/SentrySourceContextMultiModule

Expected behavior

Running assembleRelease prepares source context bundle with all classes from x and all classes from y modules.

Current behavior

Running assembleRelease prepares source context bundle with only classes from x (proof: https://github.com/wzieba/SentrySourceContextMultiModule/actions/runs/8467510900/job/23198486580#step:5:20)

Solution Brainstorm

I think it could be addressed manually with additionalSourceDirsForSourceContext, but I haven't prepared a working solution yet. Ideally, It'd be great if SAGP could do this internally (maybe as a non-default option?).

[kahest]

Hey @wzieba thanks for the suggestion, I think this makes sense, but it's not our top priority at the moment

[markushi]

Let's investigate this first on our end, as in theory it should work:

1. Apply plugin to library modules as well (and disable all other plugin features)
2. Manually upload source context via gradle
3. Verify asset merging combines all bundle IDs
4. Ensure app sends proper bundle IDs

[romtsn]

The end goal would be for our gradle plugin to support applying on the root project level, so then we could have access to all modules, but this is a bit bigger initiative

[prfarlow1]

So as of right now, if we use Sentry in a multi module Android project, we cannot have source context outside of the main app module which applies the Sentry Gradle Plugin? Meaning we also cannot use a CODEOWNERS file for ownership rules to auto-assign issues that exist outside of the app module?

[romtsn]

So as of right now, if we use Sentry in a multi module Android project, we cannot have source context outside of the main app module which applies the Sentry Gradle Plugin?

yes that's correct

Meaning we also cannot use a CODEOWNERS file for ownership rules to auto-assign issues that exist outside of the app module?

Nope, source context is not a prerequisite for code owners, stack trace linking is. However, if you have many modules, stacktrace linking is also not easy to set up, @wzieba came up with a nice solution in the other issue: https://github.com/getsentry/sentry-android-gradle-plugin/issues/546#issuecomment-2035085154

[zsperske]

I created a similar issue here: https://github.com/getsentry/sentry/issues/74435

IMHO this should be given higher priority. Most large/mature Android apps are multi-module.

We have hundreds of Gradle modules, I've heard of apps with thousands.

[abualgait]

Our organization is adopting a multi-module project structure, which directly affects the use of Sentry within the app. This should be prioritized. Thanks.

[romtsn]

We're planning to focus on this this quarter, so stay tuned for updates! We're also doing some work on the backend side to automatically infer code mappings and in-app frames for multi-module apps. This should work particularly well with our github integration.

[romtsn]

Adding another use-case here that currently doesn't work for the multi-module case: native debug symbols are only collected/uploaded only from the app module at the moment, we should expand it and run the cli command on the root project level to collect all necessary symbols from the other modules

[zsperske]

We're planning to focus on this this quarter, so stay tuned for updates! We're also doing some work on the backend side to automatically infer code mappings and in-app frames for multi-module apps. This should work particularly well with our github integration.

Are your efforts this quarter also going to address the issue with "stack trace root" and "source code root" from code mappings?

https://docs.sentry.io/product/issues/suspect-commits/#stack-trace-root-and-source-code-root

Sentry's repository integration expects a single directory for stack trace and source code roots. As covered in this issue, in a multi-module Gradle project, there is no single source or stack trace root.

[romtsn]

@zsperske yes that's precisely what we're going to focus on!

[zsperske]

While we're waiting for this functionality, I wanted to share the temporary solution I came up with. Using additionalSourceDirsForSourceContext as suggested above I was able to include all our other source code. This is definitely not the ideal way to do this as it only works for source that is part of your project. If you are in a multi-repo situation and/or are publishing versioned artifacts of your modules and then consuming them, this will not work as you want.

// in app/build.gradle.kts
sentry {
   // Other config
  additionalSourceDirsForSourceContext.set(findSourceDirectories())
}

fun findSourceDirectories(): Set<String> {
  val appModulePath = project.projectDir
  val sourceDirectories = mutableSetOf<String>()

  rootProject.subprojects.forEach { subproject ->
    if (subproject.projectDir == appModulePath) return@forEach

    val srcDirs = subproject.projectDir.walkTopDown()
      .filter { it.isDirectory && it.name == "src" }
      .mapNotNull { srcDir ->
        val mainJava = File(srcDir, "main/java")
        val mainKotlin = File(srcDir, "main/kotlin")
        when {
          mainJava.exists() -> mainJava
          mainKotlin.exists() -> mainKotlin
          else -> null
        }
      }
      .map { srcDir ->
        // Convert the absolute path to a relative path from the app module
        appModulePath.toPath().relativize(srcDir.toPath()).toString()
      }

    sourceDirectories.addAll(srcDirs)
  }

  return sourceDirectories
}

[romtsn]

@zsperske thanks for sharing, this looks like a viable workaround for now. There's one other downside that it most likely won't work with the Isolated Projects feature of Gradle (which probably becomes a default sooner-or-later).

[armenzg]

This is the ticket on creating code mappings automatically for Java projects with GitHub integrations: https://github.com/getsentry/sentry/issues/79888

I will start looking into it next month. If any of you want to be early testers let me know over there.

[armenzg]

My system is ready to be tested.

If you're a developer who has the GitHub integration and a Java-based project, please mention your org slug (or org ID) in ticket #79888, and I will opt you in.

The way this system works is: as events come in, if we find the files that your frames represent in one of your GitHub projects, we will mark the frame as in-app. This will add stack trace links to your code (see screenshot):

The added benefit is that Sentry's ML grouping will improve its accuracy.