bazelbuild
Inefficient caching of Android binary artifacts Vs libraries/tests.
Description of the feature request:
TL;DR; In order to properly populate remote cache for developers, one has to build both android binaries and android/jvm libraries. This is costly and inefficient, as it requires double CI capacity/jobs. Can we please fix this?
When building android binaries, target platform is changed to android and artifacts are not shared when, let's say, a developer build the corresponding android library(ies) to run locally a unit test. In this case, the target platform is the local machine. This means that, in order to have cache hits for both building project android binaries and project libraries/unit test/robolectric tests, we need to build both separately on CI. As these are not sharing output artifacts, jobs takes twice longer. Our CI cost essentially is doubled.
Fixing this means for most company significant $ savings. This problem is exacerbated by the fact we need to run build separately on both linux and mac.
What underlying problem are you trying to solve with this feature?
Avoiding having to rebuild project twice on CI when populating remote cache for local devs.
Which operating system are you running Bazel on?
Mac/Linux
What is the output of bazel info release?
No response
If bazel info release returns development version or (@non-git), tell us how you built Bazel.
No response
What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?
No response
Have you found anything relevant by searching the web?
Found an undocumented flag 'experimental_platform_in_output_dir' which could potentially help.
Any other information, logs, or outputs that you want to share?
No response
One half of this is on the Bazel 7 roadmap, the other one is probably covered by https://github.com/bazelbuild/bazel/issues/6526. Both are actively being worked on.
Yes, this should be working with 7.2.0, I'll explain the details in a discussion some time next week.
@ahumesky : bumping this issue for visibility. After our chat at BazelCon2024 you said we might be able to fix this with a transition? cc @nkoroste
You can't share the Java artifacts between an android_library built in an Android configuration an android_library built in a Linux configuration. One part of this is that they are compiled with two different bootclasspaths. So the inputs to the actions are different based on the platform.
The bootstrap classpath should be extracted from the same JDK though, the one that runs on the exec platform (JDKs don't have any target constraints set). Cache sharing should thus be possible by following the steps for Java in https://github.com/bazelbuild/bazel/discussions/22658.
The bootclasspath is not the same though a big part of it is derived from the JDK. There's a different set of Java APIs available on Android devices compared to Linux JVMs. That's a big part of what this change was about.