bazelbuild
`--incompatible_compact_repo_mapping_manifest`
Motivation
The size of the <binary>.repo_mapping file linked into every runfiles tree is currently quadratic in the largest number of repositories provided by a single module extension. This can result in very large text files (100s of MBs) that are costly to write, store and upload to RBE.
Description
With the flag enabled, <binary>.repo_mapping contains
+deps+*,aaa,_main
+deps+*,dep,+deps+dep1
+deps+*,dep1,+deps+dep1
+deps+*,dep2,+deps+dep2
+deps+*,dep3,+deps+dep3
instead of
+deps+dep1,aaa,_main
+deps+dep1,dep,+deps+dep1
+deps+dep1,dep1,+deps+dep1
+deps+dep1,dep2,+deps+dep2
+deps+dep1,dep3,+deps+dep3
+deps+dep2,aaa,_main
+deps+dep2,dep,+deps+dep1
+deps+dep2,dep1,+deps+dep1
+deps+dep2,dep2,+deps+dep2
+deps+dep2,dep3,+deps+dep3
...
for the deps module extension, thus reducing the size to be linear in the number of repos.
Incompatible Flag
--incompatible_compact_repo_mapping_manifest
Migration Guide
Users don't need to take action.
Runfiles libraries have to add support for the new entries, which can be unambiguously detected by the terminal * character, which is otherwise illegal in canonical repository names. Libraries are heavily discouraged from relying on the particular structure of canonical names to implement the prefix match and should instead use a sorted map or list for lookups. They can assume that no source repo prefix is a prefix of any other source repo prefix or full name.
The old format is a subset of the new format and such will continue to be supported by runfiles libraries that have been updated to support the new format.
In which Bazel LTS version will this incompatible change be enabled?
Bazel 9
Additional Context
No response
TODO List
Update the following runfiles libraries:
- [x] https://github.com/bazelbuild/rules_shell/pull/32
- [ ] https://github.com/bazelbuild/rules_cc/pull/417
- [x] https://github.com/bazelbuild/rules_java/pull/301
- [ ] Python
- [ ] https://github.com/bazel-contrib/rules_go/pull/4375
Other libraries will have to be updated by the community.
@aignas @rickeylev I'm not much of a Python person, would you be willing to take on @rules_python//python/runfiles along the lines of https://github.com/bazelbuild/rules_java/pull/301?