Cache request metadata is missing when using `--remote_download_toplevel`

[bduffany]

Description of the bug:

When using --remote_download_toplevel, the cache request metadata sent to the remote cache backend is occasionally missing (particularly the action_mnemonic and target_id fields here). This prevent's BuildBuddy's cache debug UI from being able to show the target and action mnemonic associated with an individual cache request, and we are instead falling back to showing an un-friendly action digest (since that's all the information available to us).

After adding some logging to a bazel development build, it appears it's because the ActionExecutionMetadata is null here: https://github.com/bazelbuild/bazel/blob/d3df5eff49c18bce9f6340e8f6eca3fa2dbd8dcc/src/main/java/com/google/devtools/build/lib/remote/RemoteActionInputFetcher.java#L83

Is it possible to get the ActionExecutionMetadata here based on the action ID associated with the FileArtifactValue, and pass that in when creating the RequestMetadata, instead of null? I'd like to submit a patch, but it'd be super helpful to know whether someone already looked at this before and determined that it's infeasible (for example, if it's too expensive to get that info).

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

You can reproduce it by building a bazel development build and then running

# Clone the buildbuddy-io/buildbuddy repo, as a test repo.
cd /tmp/ && git clone https://github.com/buildbuddy-io/buildbuddy && cd buildbuddy

# Try building //server, with a remote cache and remote_download_toplevel.
# Replace bazel-dev with the path to your bazel development build.
bazel-dev build //server --config=cache --remote_download_toplevel

Then open the BB UI in the invocation link that is printed out, open the "Cache" tab, then sort by Download size (desc), Show all, Group by none. Note that the action mnemonics are missing from the left hand column, and action digests are shown instead.

Which operating system are you running Bazel on?

Ubuntu 20.04

What is the output of bazel info release?

development version

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

bazel build //src:bazel-dev from master.

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

https://github.com/bazelbuild/bazel
d3df5eff49c18bce9f6340e8f6eca3fa2dbd8dcc
d3df5eff49c18bce9f6340e8f6eca3fa2dbd8dcc

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[brentleyjones]

cc: @coeuvre @tjgq

[coeuvre]

actionId stored in the metadata is the ID (in terms of remote-api) of action that generates the output. It's not easy to map this id directly to Bazel's action. Not to mention we probably will remove this from metadata https://github.com/bazelbuild/bazel/pull/11236#issuecomment-1282274643.

This leaves us the only option (AFAICT) to somehow pass in the action reference when calling this method. However, the design of the prefetcher allows Bazel to download files without an action being around which means you will always have the case that cache requests are sent without action metadata.

[github-actions[bot]]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

[jfirebaugh]

Still relevant.

[coeuvre]

This should be fixed by https://github.com/bazelbuild/bazel/commit/3b39ab8e0b0cc91a4970141eaa3469047076dcca in Bazel 7.

[jfirebaugh]

Confirmed, I can now see request metadata in Buildbuddy for all CAS hits/writes. Thank you so much for fixing this!