Provide support for releasing Starlark rules

[hlopko]

Hello EngProd :)

I see there is a space in the Bazel ecosystem for leadership regarding releasing. Currently we have quite a few repositories that will eventually be members of the federation that don't have releases (rules_cc, rules_java, platforms, rules_foreign_cc ...). Then we have other rules that do have releases, but they've all built their own homegrown tooling around releasing (e.g. rules_go). Benefits:

• No duplicated efforts, rules owners don't have to invent their own tooling
• Removed chaos, since all rules will converge on the same solution
• Experts are owners (you :), resulting in better integration with the Bazel CI

Since I come from the 'we-don't-yet-have-releases' camp, I might be oversimplifying things, but what I think we need is:

• Build the release archive with Bazel
  • Only when the whole test suite is green
  • Potentially with a run of an equivalent of Bazel's downstream pipeline
  • This should be deterministic - e.g. zeroing out timestamps in the archive.
• Upload the archive to Github
• Upload the archive to mirror.bazel.build
• Generate updated WORKSPACE rule snippet (e.g. https://docs.bazel.build/versions/0.26.0/skylark/deploying.html#readmehtml)
  • Extra point for creating a PR with updated README.

How and where in the process should we deal with release notes?

Thank you all!

[hlopko]

CC @jayconrod for more ideas :)

[hlopko]

cc @philwo

[hlopko]

cc @aiuto @dslomov

[fweikert]

EngProd already has plans to add release capabilities to Bazel CI in Q3, especially motivated by Bazelisk, Buildtools and Bazel itself. We're happy to address your use cases, too :)

[hlopko]

\o/ :)

[jayconrod]

@fweikert What are the plans?

In case it's useful, rules_go has a release process but little automation. To summarize:

• We maintain two or three stable release branches. I cherry-pick support for new Go versions, bug fixes, and compatibility updates to those branches. No API changes or updates to dependencies are made on those branches.
• Release archives are packaged with git archive and uploaded to GitHub and bazel-mirror.
• Release notes are written manually. In theory this is automatable, but it would require PRs to be a lot more organized. Probably too complicated.
• Boilerplate in README files is updated manually. This is also automatable, but it doesn't take much time.

[aiuto]

I've been exploring packaging techiniques over the last few weeks. I finding myself in favor of not using full archives for releases. Instead we should be providing releases in two packagings.

1. Package for rule users
   • Just the BUILD, .bzl and helpers needed to use the rules.
   • a deps.bzl that includes the functions <rule_name>_dependencies() and <rule_name>_register_toolchains(). If either is not needed, a no-op implementation should exist. That makes the pattern of using any rules the same.
2. The archive of the entire snapshot tag.
   • This can be achieved simply doing a github release. Github synthesize those automatically.

The first package is interesting because it means you will probably have to refactor rules so that the tests and other packaging are distinct from the core rules. The reason for doing this is to prevent accidental bloat - where your regression tests on your rules bring in more than what is usually needed - possibly even a new language toolchain. At a minimum, complete rules will depend on skydoc to produce documentation, but users do not actually need the docs in practice. You should not have to bring in skydoc just to use the rules.

I have started to organize the pkg_tar/deb/rpm rules that way.. https://github.com/bazelbuild/rules_pkg/tree/master/pkg/distro I am also working on some tools to help generate the text of the release notes page (same folder). One important part is to generate an accurate stanza for someone to include in their WORKSPACE. It should have the correct URL and sha256 values. E.g. https://github.com/bazelbuild/rules_pkg/releases

I will continue to iterate on the release packaging code in rules_pkg until I am happy with it, but it could also go in its own repository.

[jayconrod]

About packaging for rule users: I've been thinking about something like this for integration testing. There could be some target that packages rules and supporting tools and libraries, then extracts those in a test workspace, invokes bazel commands in that workspace, and verifies the output. I hadn't thought about using that for release artifacts, but it makes sense.

About archives of GitHub tags: be careful not to rely on the SHA-256 sum of GitHub's auto-generated release archives. They may change when GitHub upgrades git, zip, or whatever else. This happened a couple times in 2017, and it broke everyone on old versions. I think it's better to attach the output of git archive to the release or upload to bazel-mirror.

[hlopko]

@aiuto - nit - https://docs.bazel.build/versions/master/skylark/deploying.html#registering-toolchains suggests the name `<rule_name>_toolchains(). Not that it matters, let's just be consistent (feel free to edit docs if you care strongly).

I was already planning to not include non-rule related deps and srcs in the release archive, so package 1. I'm all in.

What is 2. archive for? Who will use it? I was not planning to release this package, so I probably miss some use case.