bazelbuild
Design a best practice for putting "change #" in an output filename
Paraphrased from #291
We have rules internally in vmware that put "change numbers" in file names. Some observations from experience
- this is error prone on CI systems that do incremental builds. Changing file names will cause files to accumulate over time, filling up disk space.
- if some piece of CI does not know the parameter passed into the build it may have to heuristically choose the "latest" file found with a glob. Which make me :( .
We have been batting around the idea of "pkg_install" that would produce an "install script" that would copy (or link) from bazel-out file locations derived from targets to a directory outside of bazel-out, renaming as it goes. Ideally the pkg_install script would be the only place that would need to know about command line parameters that change file names and we would not need this capability on all of the pkg_* rules.
@mzeren-vmw @nacl
What we do at Google separates the concerns of continuous testing vs. integration. We specify a package as
pkg_rpm(name="foo", version="@DATE@", release="@CHANGE@", ...)
and let the continuous test system build with the strange names. We don't fill up cache with new names all the time. If a user builds locally, they see @DATE@ rather than a real date. We don't let users create valid releases (or release candidates) from code that is not checked in, so we don't need the local build or test systems to produce high quality package names.
releases are more intentional. The release system
- lets the user specify the change to build from
- injects the date and change # into the build environment
- asks a secure build system to build and sign the release from checked in code.
This split also avoids the fragility you describe about some piece of CI not knowing what to do. We can write multi-binary integration tests that simply refer to components as foo-@DATE@-@CHANGE@ and they will get what was just built for the test. We don't have to go through each test to be able to parameterize the names of binaries our test harness runs.
When we get around to this, we might consider if package_file_name expansion could support a default, like a shell escape. So
package_file_name = "foo-${COMMIT-head}"
What people might really want is a conditional, so that the file name is either "foo" or "foo-PR. Something like
package_file_name = "foo-$(if COMMIT -${COMMIT})
I would be very resistant to adding that complexity. If people want that, they could make the COMMIT variable being with a '-', and do package_file_name="foo${COMMIT}"