rules_closure icon indicating copy to clipboard operation
rules_closure copied to clipboard

`filegroup_external` should set `canonical_id`

Open wchargin opened this issue 4 years ago • 0 comments

Using filegroup_external to download files has the downside that if one bumps the version number of the source URL, but forgets to update the SHA-256 checksum, then Bazel will assume that the content is unchanged and will silently use the old version. The canonical issue thread for this is: https://github.com/bazelbuild/bazel/issues/5144

Bazel now exposes a way to mitigate this problem—ctx.download takes a new argument canonical_id, which forms part of the cache key: https://github.com/bazelbuild/proposals/blob/master/designs/2019-04-29-cache.md

The filegroup_external rule could set the canonical_id to (a lossless encoding of) the whole list of URLs, such that any change to the list would cause a re-download. This has the slight downside that adding a new URL to an existing list purely to increase redundancy would also cause a one-time cache miss, which seems acceptable.

This would have saved a fair amount of confusion on the following PR: https://github.com/tensorflow/tensorboard/pull/2898

wchargin avatar Nov 06 '19 22:11 wchargin