bazel
bazel copied to clipboard
bazelbuild
Remote executor used in tests does not support dangling symlinks
When attempting to remotely execute an action that produces a dangling symlink, the remote executor throws an exception here: https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/remote/UploadManifest.java;l=194;drc=93029d8f1f01be440d8e06cce9585ad912f9169b
This must be fixed before integration tests for dangling symlinks can be added. This may be a blocker for #10298.
I think the more important point is whether RBE itself supports unresolved symlinks; to my best understanding, yes. I would be okay with releasing Bazel 6.0 without that bit of test infrastructure, if necessary, but it would be better not to do so.
So let's make this, P2 and a release blocker, with maybe revisiting the latter it if we are in a time crunch?
The remote execution spec definitely supports actions that create a symlink, but it doesn't say whether the symlink must point to an existing file or directory or not (at least according to my reading). The relevant protocol fields are output_file_symlinks, output_directory_symlinks and output_symlinks. Note that the former two are deprecated in favor of the latter, but Bazel follows the older spec.
The Google Cloud RBE implementation, as far as I can tell, silently fails to report a dangling output symlink. I don't know about other implementations. A minimal repro is at https://github.com/tjgq/bazel-rbe-unresolved-symlink (but note that, even if the RBE implementation worked, the build would still fail on the Bazel side due to #16290).
Oof. For some reason, I thought that the Google's version of remote execution does handle both input and output symlinks. Then there is no alternative than to implement it in our mock backend if we want to test the functionality and we should definitely do so.
I've confirmed with folks who work on Google RBE that output symlinks are not currently implemented (specifically, symlinks that resolve successfully are reported as the regular files or directories that they resolve to, and dangling symlinks are omitted from the outputs entirely). So we do need to test against the mock backend.