bazel-skylib
bazel-skylib copied to clipboard
Add helper rule/provider for working with files inside tree artifacts
TreeArtifacts have a lot of shortcomings. One of them is that it's impossible for an action to produce labels both for a directory output (ctx.actions.declare_directory
) and also a label for a file within that directory. (some parts of Bazel fail to strictly disallow this, but at least the RBE protocol fails when you do this.)
In rules_nodejs we are working on providing dependencies as directories. This is because in the npm ecosystem there's no equivalent of a .jar file, so our dependencies are exploded and easily reach 10k input files to an action, making sandbox setup time a significant portion of action execution time.
However there's no accepted way across the Bazel ecosystem to continue referencing one file within a directory. One use case where you need this in many languages is the "main". Take https://docs.bazel.build/versions/master/be/python.html#py_binary.main for example - if you wanted to supply a directory (TreeArtifact) to the srcs
of a py_binary
you'd have no way to say which file within that directory should be executed.
We'll probably invent our own solution just for rules_nodejs (https://github.com/bazelbuild/rules_nodejs/pull/2579) but we'd rather upstream this concept into skylib so that rulesets can be more interoperable when they need to pass each other a construct of type {Label of TreeArtifact, string path within that label}
Update, we did invent a rules_nodejs-specific rule https://github.com/bazelbuild/rules_nodejs/blob/7f688a95c9002a9e5148f218dfa9f1ea6464f50a/nodejs/private/providers/directory_file_path_info.bzl but would still be better to share this, since it has a Provider symbol that ought to allow interop between rulesets.
Update again, this is now in bazel-lib: https://docs.aspect.build/aspect-build/bazel-lib/v0.9.6/docs/directory_path-docgen.html
Such a thing would be useful to standardize, whether in code or at least in documentation. @comius - what do you think?