bazel-buildfarm icon indicating copy to clipboard operation
bazel-buildfarm copied to clipboard

Implement support for remote cache eviction support for build w/out the bytes

Open shirchen opened this issue 2 years ago • 3 comments

With https://github.com/bazelbuild/bazel/pull/16660 merged, bazel now keeps track of TTLs of entries that it assumes remote cache will have.

For buildfarm, for FindMissingBlobs call, we need to create a mechanism to try and not evict these entries since client will assume they are there for duration of experimental_remote_cache_ttl value.

Quote from design doc: https://docs.google.com/document/d/1wM61xufcMS5W0LQ0ar5JBREiN9zKfgu6AnHVD7BSkR4/edit#

During the invocation, it periodically sends FindMissingBlobs request to the remote cache with the blobs, hoping that the server will increase the lifetimes of referenced blobs. The frequency is based on --experimental_remote_cache_ttl.

shirchen avatar Jun 07 '23 20:06 shirchen

There's a hitch here: The client isn't saying anything about the actual ttl value to the remote cache.

We can probably assume that a default coupled pair of client/remote cache is 3h, and make it configurable, but there's no guarantee that we can make for a single client that lengthens that ttl, because it won't be reflected on the server side.

Aside: The mechanics of the frequent findMissingBlobs calls from the bazel client are not obvious to me based on the current state of their source tree, and that referenced PR is not actually merged: the last comment associated with it indicates that it was accomplished with other smaller PRs. I will have to discover them to get a clear picture of what is actually performing the refreshes.

werkt avatar Jun 11 '23 13:06 werkt

Should it be user's responsibility to set the same value in both bazel and buildfarm. In this case it could be experimental_remote_cache_ttl and casExpireKey.

Also, can we just reset the expiry of digest key on direct access (read) or its reference access (getac/fmbs). Expire reset value can be same as casExpireKey configured in config. Also this make cache LRU.

amishra-u avatar Jun 15 '23 03:06 amishra-u

See https://github.com/bazelbuild/bazel/issues/22592 (--experimental_remote_cache_ttl=0s breaks --remote_download_outputs=toplevel (Build without the Bytes))

dws avatar May 30 '24 18:05 dws