OSS dynamic feature support

[mauriciogg]

Abstract

This issue describes the current state of dynamic features in Bazel, the differences between Bazel and Gradle as well as the required work to bring them to parity with Gradle to support Open Source builds. This document focuses on the two main issues with the current version of the rules: native library support and dex support. Finally, an appendix with all the issues we encountered can be seen at the end. These issues are minor and are only listed for documentation purposes.

Current support in Bazel:

Currently Bazel supports dynamic features via the android_feature_module and android_application rules. In order to create a feature module, you declare a new android_feature_module target and pass it to an android_application target via the feature_modules attribute. The android_feature_module can’t be used on its own. It must always be used through android_application.

The android_feature_module rule supports creating dynamic features that contain native libraries and/or assets only. There is no support for dynamic features that contain Java/Kotlin code or Android resources.

In order to specify the dependencies of a dynamic feature, one can either pass an android_library target through the library attribute or/and an android_binary target through the binary attribute of the rule. The full list of attributes the rule takes can be found here.

The assets for the dynamic feature are collected from the set of transitive dependencies of the dynamic feature (via library and binary attrs).

The native libraries the dynamic feature are collected from the APK of the Android Binary target passed via the binary attr of the feature module rule. The android_application rule extracts all the libraries in the binary apk and packages them with the module. I.e. the module will include all the libraries in the transitive dependencies of the binary.

Dynamic feature support in Gradle {#dynamic-feature-support-in-gradle}

Gradle supports dynamic features via the dynamicFeatures attribute. The base module is added as dependency of every feature module by default in the build.gradle file (not this is fundamentally not possible in Bazel due to the cyclic dependency). The module will contain the direct dependencies specified in the build.gradle. Only artifacts that are explicitly specified in the build.gradle file are considered to be part of the feature module. The dependency on the base module is only used for compilation purposes. This means that multiple feature modules can have common dependencies without ending up with multiple duplicate dependencies in different modules.

Assets, native libraries, Java/Kotlin code and resources are supported.

Issues with current Bazel approach

The current approach used by Bazel to build feature modules, while simple, has issues when building complex feature modules that include more than just assets.

The native libraries support relies on building an intermediate APK which is only used to extract the .so files. This binary can’t be reused across feature modules, since doing so would mean duplicating the native binaries included in each feature module. In addition to that, this future is implemented by inspecting the IdeInfo provider in order to detect if the target includes native binaries, which seems like a hacky use of the provider.

The same approach could be used in order to support Dex files (create a binary target and extract the dex files from the APK) but this would also fall into duplicate dependencies being included in each feature module in any case where different feature modules share common dependencies. If we wanted to use this approach, we’d need to take care into crafting targets with a disjoint set of dependencies (which is infeasible if we want to share code) or craft custom targets marked as neverlink in order to be used by the feature module. Doing this would mean writing a lot of boiler plate targets and could prove to be very error prone.

Changes to android_application and android_feature_module rule

In order to enable dynamic features with Bazel in Gradle repos with minimal modifications to the current builds working with Gradle, we propose flattening the native_feature_module rule and allow users to explicitly specify the dependencies that need to be included (android_library and native library deps). Concretely we propose replacing the library and binary attributes for a deps attribute.

The deps attribute would take kt_android_library, android_library and aar_import targets. For dependencies with code it will just dex the code just in that dependency (it will not dex the set of transitive deps) and will collect any native library included in that dependency (no transitive libraries as well). In order to enable dexing, the code from DexArchiveAspect will need to be lifted into starlark and applied at the library level instead of the binary level (most of the code can be simplified since there is significant legacy code that can be ignored).

One remaining issue with this approach is that R8/proguarding feature modules requires the full set of binary dexes and feature module dexes. Currently the R8 code lives in the native android binary so any support for R8 with feature modules is TBD until there is more clarity on the future of the android_binary rule.

[nkoroste]

cc @timpeut

[ted-xie]

Hi Mauricio,

Thank you for bringing this feature request to our attention! Overall what you are asking for makes sense, and the design strategy you’ve proposed seems sound in the context of a Bazel ruleset. However, due to the nature of the changes requested to the base rules_android codebase, even though they seem relatively small, we cannot implement this type of change in google3, which is the source of truth for rules_android itself.

A core tenet of this feature request is that the current level of feature module support is insufficient for the author’s use case. While the incompatibility with your app’s technical requirements is unfortunate, internal rules authors have deliberately designed the feature this way. There are no Google-internal use cases that require anything other than assets in feature modules; we simply do not have any apps that require more than this level of support. Changes to the API surface of build rules require significant vetting, approval, and (slow) rollout internally, so we would prefer to leave the API as-is whenever possible. Therefore, we propose an alternative solution to this problem.

Dynamic feature modules and feature splits in Google3

Even within Google3, the rules_android codebase serves only as a baseline rule set that provides a basic set of features for Android builds and tests. We have designed the rules_android codebase to be modular such that rules such as android_binary and android_library can be easily imported into a wrapper and extended with additional processor pipeline stages, additional attributes, or other features as required. This is already an established design pattern in Google3, and AOSP has already adopted this pattern. See here for an example of some simple modifications to the android_binary processing pipeline.

Alternative proposal from the Bazel Team

We propose that instead of modifying rules_android, which as mentioned earlier comprises just the set of core functionality for Android builds, that any OSS users who desire to change rule attribute behavior do so in various wrapper macros and processor pipeline modifications. The above AOSP repository link should serve well as a demonstration of the kind of design pattern we use internally.

Benefits of the Bazel Team’s proposal

Overall, our proposed implementation strategy here should be much cleaner than directly modifying rules_android for the desired feature module support. For the Bazel team, it means avoiding accepting code that will never be used, and thus will impose an undue maintenance burden upon all of the stakeholder teams. For OSS/Bazel users, it means that any existing forks will not have to diverge even further from the existing codebase shared on Github, and it will be much easier for the Bazel team (as well as community users) to support and triage any issues that do arise from the core set of rules_android features.

[mauriciogg]

For the Bazel team, it means avoiding accepting code that will never be used, and thus will impose an undue maintenance burden upon all of the stakeholder teams.

Hi Ted,

Thanks for your response. The wrapper approach makes sense, but I still think it would be valuable to have this wrapper upstreamed even if its not imported back into google3 and not used inside google so the community can share a single implementation for dynamic features. Otherwise, we might end up in a situation where each company maintains different versions of the same thing. I don't know if this is the right place to discuss this, but maybe we could implement OSS features only like these wrappers in a contrib/ folder or something like of the sorts that can be easily excluded by copybara.