rules_proto_grpc icon indicating copy to clipboard operation
rules_proto_grpc copied to clipboard

Add alternate download mirror

Open aaliddell opened this issue 3 years ago • 0 comments

Description

Given GitHub has had uptime issues recently, it has raised awareness on the dependency of Bazel builds on the availability of GitHub. Rulesets on the official Bazel org use mirror.bazel.build to have an alternate download URL.

Since the domain rules-proto-grpc.com is already used for this project, something like mirror.rules-proto-grpc.com could be implemented. Users do not need to explicitly trust a mirror, since Bazel is already checking hashes on download to ensure integrity.

Ideally this would be done in a privacy preserving way, since I don't need nor want to track downloads through this mirror. However, realistically any service used to implement this will have access logging that I cannot disable provably to users, so the implementation of this mirror should at least be open to allow users to choose whether they wish to use it or not.

TODO:

  • Find mirror hosting solution that is simple to maintain, secure and cost effective. At the most minimal this could be S3-like-bucket + CDN. However, since GitHub gives no guidance on assets download counts, it's hard to gauge the required bandwidth for offering this mirror, so services with higher egress costs (the 'clouds') are a cost risk.
  • Determine Bazel behaviour when given multiple URLs. Is it random choice, or primary/fallback? Likely former.
  • Add workflow to push .tar.gz generated by GitHub to alternate data store. Alternatively, generate .tar.gz manually and attach to GH release (see rules_nodejs etc), which allows dropping example dirs etc that are not needed by end users. This will be the workflow likely required for bzlmod anyway in the future.
  • Backfill mirror with existing releases (optional)
  • Update docs generation to include alternate URLs
  • Ensure we use alternate mirrors wherever available in repositories.bzl

See also: https://github.com/bazel-contrib/SIG-rules-authors/issues/12. Since this may be solved naturally by use of BCR, it may be preferrable to just wait for bzlmod...

aaliddell avatar Nov 28 '21 19:11 aaliddell