cmake: file installed from the source directory are only symlinked

[t-8ch]

When using cmake() together with Bazel 7.0 with exported headers the build fails.

$ cd rules_foreign_cc/examples
$ USE_BAZEL_VERSION=7.0.0rc7 bazelisk build //cmake_hello_world_lib/shared/... 
INFO: Analyzed 3 targets (0 packages loaded, 0 targets configured).
ERROR: /home/me/src/rules_foreign_cc/examples/cmake_hello_world_lib/shared/BUILD.bazel:13:6: Error while validating output TreeArtifact File:[[<execution_root>]bazel-out/k8-fastbuild/bin]cmake_hello_world_lib/shared/libhello/include : Failed to resolve relative path hello.h inside TreeArtifact /home/tweisssc/.cache/bazel/_bazel_tweisssc/691c198c0f7498b76c679347f393fea9/execroot/_main/bazel-out/k8-fastbuild/bin/cmake_hello_world_lib/shared/libhello/include. The associated file is either missing or is an invalid symlink.
ERROR: /home/me/src/rules_foreign_cc/examples/cmake_hello_world_lib/shared/BUILD.bazel:13:6: Foreign Cc - CMake: Building libhello failed: not all outputs were created or valid
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.428s, Critical Path: 0.36s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
ERROR: Build did NOT complete successfully

And indeed the installed header file is a symlink:

$ ls -l /home/me/.cache/bazel/_bazel_me/691c198c0f7498b76c679347f393fea9/execroot/_main/bazel-out/k8-fastbuild/bin/cmake_hello_world_
lib/shared/libhello/include/hello.h
lrwxrwxrwx 1 me me 66 Dec 11 16:57 /home/me/.cache/bazel/_bazel_tweisssc/691c198c0f7498b76c679347f393fea9/execroot/_main/bazel-out/k8-fastbuild/bin/cmake_hello_world_lib/shared/libhello/include/hello.h -> /tmp/bazel-source-roots/0/cmake_hello_world_lib/shared/src/hello.h

The problem is that cmake will preserve symlinks when installing them instead of resolving them to the contents (see cmFileCopier::Install()). cmake() with filegroup() symlinks all the sources into the internal build tree, leading for example headers from the source tree to be also installed as (broken) symlink.

[t-8ch]

See also https://github.com/bazelbuild/bazel/issues/10299

[cameron-martin]

This is likely the issue I was seeing here: https://bazelbuild.slack.com/archives/CGA9QFQ8H/p1701343519010999

[pjjw]

for those that get here and aren't on the bazel slack, the temporary workaround is the flag --noincompatible_sandbox_hermetic_tmp

[t-8ch]

There are two workarounds we tested:

1. Patch the original CMakeLists.txt to copy the source file to the build directory with ${CMAKE_COMMAND} -E copy and then install that copy, so it won't be a symlink anymore.
2. Run a script over the install directory after the installation to replace symlinks pointing outside of the install directory with a copy of the file they are pointing to.

Solution 2. Could also be implemented natively by rules_foreign_cc in a generic way.

[pjjw]

if feedback from the peanut gallery is welcome, option 2 seems like the option least-likely to have baffling corner cases. we're already doing a bunch of backbends with rules_foreign_cc involving multiple cmake projects reading config from each other, so any straightforward patching approach probably wouldn't work for our admittedly goofy usage.

[jsharpe]

https://github.com/bazelbuild/bazel/pull/20603 - this may fix this issue too

[t-8ch]

@jsharpe It does not fix the issue for me. The same error happens and the same sandbox-relative symlink appears in the cache.

[eguiraud]

for those that get here and aren't on the bazel slack, the temporary workaround is the flag --noincompatible_sandbox_hermetic_tmp

for the record, the slightly less scary --sandbox_add_mount_pair=/tmp also seems to be a workaround.

[pjjw]

that has the same effect as --noincompatible_sandbox_hermetic_tmp, it is identically scary :)

[cameron-martin]

Possibly this'll fix this issue, and is marked for 7.1: https://github.com/bazelbuild/bazel/issues/21215

[criemen]

For the record,

common --noincompatible_sandbox_hermetic_tmp
common --sandbox_tmpfs_path=/tmp

worked for me on Bazel 7, while (presumably) still preserving the hermetic tmp directory which is necessary for us to address occasional hotspot crashes during Java compilations.

[TendTo]

There are two workarounds we tested:

1. Patch the original CMakeLists.txt to copy the source file to the build directory with ${CMAKE_COMMAND} -E copy and then install that copy, so it won't be a symlink anymore.
2. Run a script over the install directory after the installation to replace symlinks pointing outside of the install directory with a copy of the file they are pointing to.

Solution 2. Could also be implemented natively by rules_foreign_cc in a generic way.

Speaking of the second workaround, I thought I may provide the current implementation I'm using, in case someone needs it.

# Just add the 'postfix_script' to ensure that symbolic links are copied from the original files, removing the dangling problem
cmake(
    name = "external_lib",
    lib_source = ":all_srcs",
    # ...
    postfix_script = """find $INSTALLDIR -type l -exec sh -c 'for i in "$@"; do cp --preserve --remove-destination "$(readlink -f "$i")" "$i"; done' sh {} +""",
)

filegroup(
    name = "all_srcs",
    srcs = glob(["**"], exclude = ["bazel-*"]),
)

I am not an expert, and there may be some good reasons not to do so, but you may want to consider whether to add such a line directly into the official script