meta-clang
meta-clang copied to clipboard
Published
20 hours ago •
kraj
kraj
libclang.so.13.0.1 can not find libc.so.6 in SDK
Describe the bug When adding libclang to SDK and used in a build the dynamic linker will use libc.so.6 from the system instead of the one from the SDK. There can be a version
This is found when trying to build rust code with librocksdb-sys using bindgen. Error from rust:
...
Running `rustc --crate-name build_script_build --edition=2018 /work/sdk/poky-x86/sysroots/core2-64-poky-linux/home/cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="static"' -C metadata=7c8525cf0d81b61c -C extra-filename=-7c8525cf0d81b61c --out-dir /work/tmp/rust-hello/target/release/build/librocksdb-sys-7c8525cf0d81b61c -L dependency=/work/tmp/rust-hello/target/release/deps --extern bindgen=/work/tmp/rust-hello/target/release/deps/libbindgen-466101c122bb103a.rlib --extern cc=/work/tmp/rust-hello/target/release/deps/libcc-17b88774702fcf3c.rlib --extern glob=/work/tmp/rust-hello/target/release/deps/libglob-b01e7b2cb01490de.rlib --cap-lints allow`
Running `/work/tmp/rust-hello/target/release/build/librocksdb-sys-7c8525cf0d81b61c/build-script-build`
error: failed to run custom build command for `librocksdb-sys v6.20.3`
Caused by:
process didn't exit successfully: `/work/tmp/rust-hello/target/release/build/librocksdb-sys-7c8525cf0d81b61c/build-script-build` (exit status: 101)
--- stderr
thread 'main' panicked at 'Unable to find libclang: "the `libclang` shared library at /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1 could not be opened: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1)"', /work/sdk/poky-x86/sysroots/core2-64-poky-linux/home/cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.59.1/src/lib.rs:2117:31
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
When running ldd on libclang.so.13.0.1 and checking versions of libc.so.6:
$ ldd /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1
/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1)
/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/libLLVM-13.so)
/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/libstdc++.so.6)
/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/libxml2.so.2)
linux-vdso.so.1 (0x00007ffdf3d47000)
libLLVM-13.so => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/libLLVM-13.so (0x00007f1c7dabf000)
libstdc++.so.6 => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/libstdc++.so.6 (0x00007f1c7d8b0000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1c7d74f000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1c7d734000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1c7d548000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1c84f0d000)
libffi.so.8 => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/libffi.so.8 (0x00007f1c7d53a000)
libedit.so.0 => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/libedit.so.0 (0x00007f1c7d4ff000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1c7d4f4000)
libz.so.1 => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/libz.so.1 (0x00007f1c7d4da000)
libxml2.so.2 => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/libxml2.so.2 (0x00007f1c7d378000)
libncurses.so.5 => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/libncurses.so.5 (0x00007f1c7d34f000)
libtinfo.so.5 => /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/libtinfo.so.5 (0x00007f1c7d320000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1c7d2fe000)
$
$ /lib/x86_64-linux-gnu/libc.so.6 --version | head -n 1
GNU C Library (Ubuntu GLIBC 2.33-0ubuntu5) release release version 2.33.
$ /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 --version | head -n 1
GNU C Library (GNU libc) stable release version 2.34.
To Reproduce
- Setup standard Poky build with meta-clang. Used revisions:
Build Configuration:
BB_VERSION = "1.53.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "x86_64-poky-linux"
MACHINE = "qemux86-64"
DISTRO = "poky"
DISTRO_VERSION = "3.4+snapshot-d233a42bf3067528be18334924a4b82d8363bcf2"
TUNE_FEATURES = "m64 core2"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "master-next:d233a42bf3067528be18334924a4b82d8363bcf2"
meta-clang = "master:81dca6d47885a35ffe9c05827e83670d500b7535"
- Add rust and clang to SDK by adding this to the configuration:
TOOLCHAIN_HOST_TASK:append = " packagegroup-rust-cross-canadian-${MACHINE}"
CLANGSDK = "1"
- Build SDK
bitbake core-image-minial -cpopulate_sdk - Install SDK
- Initialize SDK and create a minimal rust reproducer
$ . /work/sdk/poky-x86/environment-setup-core2-64-poky-linux
$ cargo new rust-hello-2
$ cd rust-hello-2
$ echo 'librocksdb-sys = "6.20.3"' >> Cargo.toml
$ cat Cargo.toml
[package]
name = "rust-hello-2"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
librocksdb-sys = "6.20.3"
$
$ cargo build -v --target x86_64-poky-linux
This is probably not a meta-clang issue alone. I apologize but forgot one step in the reproducer above which is a patch to relocate ncurses libs in the SDK from /lib to /usr/lib.
$ git diff meta/recipes-core/ncurses/ncurses.inc
diff --git a/meta/recipes-core/ncurses/ncurses.inc b/meta/recipes-core/ncurses/ncurses.inc
index a0ecd8a80b..4dc02652bf 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -34,6 +34,7 @@ BUILD_CPPFLAGS += "-D_GNU_SOURCE"
# natives don't generally look in base_libdir
base_libdir:class-native = "${libdir}"
+base_libdir:class-nativesdk = "${libdir}"
# Display corruption occurs on 64 bit hosts without these settings
# This was derrived from the upstream debian ncurses which uses
There are some issue with the paths that the linker searches for libraries. Looking in to libclang.so.13.0.1 issue here I have, with help from this thread https://lists.openembedded.org/g/openembedded-core/topic/80874524#148546 , extracted a lot more information with LD_DEBUG=all. Interesting info related to libclang is this:
20319: file=librt.so.1 [0]; needed by /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/libLLVM-13.so [0]
20319: find library=librt.so.1 [0]; searching
20319: search path=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/..:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib (RPATH from file /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/libLLVM-13.so)
20319: trying file=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../librt.so.1
20319: trying file=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/../lib/librt.so.1
20319: search path=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/..:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib (RPATH from file /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1)
20319: trying file=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../librt.so.1
20319: trying file=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/librt.so.1
20319: search path=/work/tmp/rust-hello/target/debug/deps:/work/tmp/rust-hello/target/debug (LD_LIBRARY_PATH)
20319: trying file=/work/tmp/rust-hello/target/debug/deps/librt.so.1
20319: trying file=/work/tmp/rust-hello/target/debug/librt.so.1
20319: search cache=/etc/ld.so.cache
20319: trying file=/lib/x86_64-linux-gnu/librt.so.1
In the SDK most of the libraries are located in /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib but also some of them, for example librt.so.1, are located in /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/lib. The RPATH of libclang.so.13.0.1 does not include <sysroot>/lib.
$ /work/yocto/poky/build/tmp/sysroots-uninative/x86_64-linux/usr/bin/patchelf-uninative /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1 --print-rpath
$ORIGIN/..:$ORIGIN/../lib
The same seems true for libLLVM (not from this layer):
$ /work/yocto/poky/build/tmp/sysroots-uninative/x86_64-linux/usr/bin/patchelf-uninative /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libLLVM-13.so --print-rpath
$ORIGIN/..:$ORIGIN/../lib
Should $ORIGIN/../..:$ORIGIN/../../lib also be added to RPATH?
But there are also more interesting/strange things in the log from LD_DEBUG=all it reports different RPATH from libclang.so.13.0.1 in different occasions:
<snip>
20319: search path=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../glibc-hwcaps/x86-64-v4:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../glibc-hwcaps/x86-64-v3:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../glibc-hwcaps/x86-64-v2:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls/haswell/avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls/haswell/avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls/haswell/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls/haswell:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls/avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls/avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../tls:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../haswell/avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../haswell/avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../haswell/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../haswell:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/..:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/glibc-hwcaps/x86-64-v4:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/glibc-hwcaps/x86-64-v3:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/glibc-hwcaps/x86-64-v2:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls/haswell/avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls/haswell/avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls/haswell/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls/haswell:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls/avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls/avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/tls:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/haswell/avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/haswell/avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/haswell/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/haswell:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/avx512_1/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/avx512_1:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib/x86_64:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib (RPATH from file /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1)
<snip>
20319: search path=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/..:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib (RPATH from file /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1)
<snip>
20319: search path=/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/..:/work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/../lib (RPATH from file /work/sdk/poky-x86/sysroots/x86_64-pokysdk-linux/usr/lib/libclang.so.13.0.1)
<snip>
Are there is some other function tampering with RPATH when using Yocto SDK?
Can you try out #533
I have tried #533 and can not see any difference in the dependencies. Have verified that CMAKE_BUILD_WITH_INSTALL_RPATH=ON is passed in do_configure of libcxx. But I can not find any dependency between nativesdk-clang (providing libclnag.so.13.0.1) and libcxx. Should there be one?
@pberginkonsult it seems specific to rust here. Can you try a simple c/c++ example using CLANGCC and see if that works ok ? if that works then we have the issue in llvm bundled with rust