FR: include required libraries so no sysroot is needed

[garymm]

Hi! I tried this in a docker container on my macbook and it did not seem to work.

Inside the docker container:

$ uname -a
Linux 25324f773e57 6.4.16-linuxkit #1 SMP PREEMPT Thu Nov 16 10:49:20 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
$ bazel test //...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed 2 targets (40 packages loaded, 1531 targets configured).
INFO: Found 1 target and 1 test target...
ERROR: /workspaces/kv_store_template/BUILD.bazel:3:11: Linking libkv_store.so failed: (Exit 1): cc_wrapper.sh failed: error executing command (from target //:kv_store) external/llvm_toolchain/bin/cc_wrapper.sh @bazel-out/aarch64-fastbuild/bin/libkv_store.so-2.params

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld.lld: error: unable to find library -l:libc++.a
ld.lld: error: unable to find library -l:libc++abi.a
ld.lld: error: unable to find library -l:libunwind.a
ld.lld: error: cannot open external/llvm_toolchain_llvm/lib/clang/17/lib/linux/libclang_rt.builtins-aarch64.a: No such file or directory
ld.lld: error: cannot open external/llvm_toolchain_llvm/lib/clang/17/lib/linux/libclang_rt.builtins-aarch64.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

My WORKSPACE.bazel:

load("@bazel_toolchain//toolchain:deps.bzl", "bazel_toolchain_dependencies")
load("@bazel_toolchain//toolchain:rules.bzl", "llvm_toolchain")

bazel_toolchain_dependencies()

llvm_toolchain(
    name = "llvm_toolchain",
    cxx_flags = {
        "": [
            "-stdlib=libc++",
            "-std=c++23",
        ],
    },
    llvm_version = "17.0.4",
    sha256 = {
        "darwin-aarch64": "74a390bca500397cff933f9e8f3a7d1c3cf3bb89389a1c1c8ba95799f5958a64",
        "darwin-x86_64": "17f18c85272de483cfd676f0ce4bca93f2a1f9c03a5384708528716720fa01d8",
        "linux-aarch64": "a360aa784a3465a8ca85cf4a20c39153ca1be864ffa56d9ceb07293fa804457c",
        "linux-x86_64": "fde82f8507a6fb8c6626d13466cc150d21582139771bdd53bb77e766275d7e05",
    },
    urls = {
        "darwin-aarch64": ["https://github.com/dzbarsky/static-clang/releases/download/v17.0.4/darwin_arm64.tar.xz"],
        "darwin-x86_64": ["https://github.com/dzbarsky/static-clang/releases/download/v17.0.4/darwin_amd64.tar.xz"],
        "linux-aarch64": ["https://github.com/dzbarsky/static-clang/releases/download/v17.0.4/linux_arm64.tar.xz"],
        "linux-x86_64": ["https://github.com/dzbarsky/static-clang/releases/download/v17.0.4/linux_amd64.tar.xz"],
    },
)

load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains")

llvm_register_toolchains()

Outside the docker container (darwin arm64) it did seem to work.

[garymm]

Doesn't work on linux amd64 outside docker either. Very similar failures.

[dzbarsky]

Hi, sorry to hear you are having issues. The distributions created by the release process only contain the compiler binaries, they don't have those libraries (libc++/libc++abi/libunwind). The relevant code in the llvm_toolchain repo is here: https://github.com/grailbio/bazel-toolchain/blob/214fc1cc5d9010fafeaa16a67059caefb1c338aa/toolchain/cc_toolchain_config.bzl#L188-L213

We have not hit this issue because we provide an explicit sysroot (which contains these libraries), which you need anyway for cross-compilation. You could try that if you want a fully hermetic build, and/or patch the section I linked to link against the system libraries instead.

I agree that it would be nice to provide these libraries as part of the distribution in this repo, but the existing LLVM project Bazel rules don't include targets for them, so it would be a bit of work to wire those up. Will keep this issue open to track it!

[garymm]

Thanks for taking a look! Maybe things would work if I don't specify -stdlib=libc++ and I have another standard library installed on the system? In any case that renders these useless for me since I want to rely on the toolchain for the standard library. Feel free to close as WAI if you like.

[dzbarsky]

I think your ask is super reasonable and would like to support it in the future, I just don't have cycles right now to write the BUILD configuration for the extra libraries :/