clang nativesdk sysroot is not correctly set.

[Lorac]

Describe the bug When using GCC from the buildtools-extended-tarball it works well since the searched path for libraries, includes, etc are from the nativesdk sysroot. When trying to use clang as a native compile through the SDK it uses hosts libraries, includes, etc.

To Reproduce Have CLANGSDK=1 and try to use it to compile a simple program with CMake. It will use libc from the host system instead of from the sysroot.

Expected behavior clang, uses libc from the native sysroot

From gcc

$ /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/g++ -v -xc++ /dev/null
Using built-in specs.
COLLECT_GCC=/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/g++
COLLECT_LTO_WRAPPER=/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/libexec/gcc/x86_64-ekosdk-linux/10.3.0/lto-wrapper
Target: x86_64-ekosdk-linux
Configured with: ../../../../../../work-shared/gcc-10.3.0-r0/gcc-10.3.0/configure --build=x86_64-linux --host=x86_64-ekosdk-linux --target=x86_64-ekosdk-linux --prefix=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr --exec_prefix=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr --bindir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/bin --sbindir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/bin --libexecdir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/libexec --datadir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/share --sysconfdir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/etc --sharedstatedir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/com --localstatedir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/var --libdir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/lib --includedir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/include --oldincludedir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/include --infodir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/share/info --mandir=/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-ekosdk-linux/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/ --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --enable-multilib --enable-default-pie --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=x86_64-ekosdk-linux- --without-local-prefix --disable-install-libiberty --disable-libssp --enable-libitm --enable-lto --disable-bootstrap --with-system-zlib --with-linker-hash-style=gnu --enable-linker-build-id --with-ppl=no --with-cloog=no --enable-checking=release --enable-cheaders=c_global --without-isl --with-build-sysroot=/ --disable-static --enable-nls --with-glibc-version=2.28 --enable-initfini-array --enable-__cxa_atexit
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.3.0 (GCC) 
COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/libexec/gcc/x86_64-ekosdk-linux/10.3.0/cc1plus -quiet -v -D_GNU_SOURCE /dev/null -quiet -dumpbase null -mtune=generic -march=x86-64 -auxbase null -version -o /tmp/ccHNqBXa.s
GNU C++14 (GCC) version 10.3.0 (x86_64-ekosdk-linux)
        compiled by GNU C version 10.3.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/gcc/x86_64-ekosdk-linux/10.3.0/../../../../x86_64-ekosdk-linux/include"
ignoring duplicate directory "/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/gcc/x86_64-ekosdk-linux/10.3.0/../../../../include/c++/10.3.0
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/gcc/x86_64-ekosdk-linux/10.3.0/../../../../include/c++/10.3.0/x86_64-ekosdk-linux
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/gcc/x86_64-ekosdk-linux/10.3.0/../../../../include/c++/10.3.0/backward
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/gcc/x86_64-ekosdk-linux/10.3.0/include
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/x86_64-ekosdk-linux/10.3.0/include
 /usr/local/include
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/include
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/gcc/x86_64-ekosdk-linux/10.3.0/include-fixed
End of search list.

From clang:

/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/x86_64-eko-linux/x86_64-eko-linux-clang++ -v -xc++ /dev/null
clang version 12.0.0 (https://github.com/llvm/llvm-project d28af7c654d8db0b68c175db5ce212d74fb5e9bc)
Target: x86_64-eko-linux
Thread model: posix
InstalledDir: /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/x86_64-eko-linux
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
 "/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/clang-12" -cc1 -triple x86_64-eko-linux -emit-obj -mrelax-all --mrelax-relocations -disable-free -disable-llvm-verifier -discard-value-names -main-file-name null -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -tune-cpu generic -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/clang/12.0.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/backward -internal-isystem /usr/local/include -internal-isystem /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/clang/12.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /home/maxime/devel/parisapps -ferror-limit 19 -stack-protector 2 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -o /tmp/null-1e8967.o -x c++ /dev/null
clang -cc1 version 12.0.0 based upon LLVM 12.0.0 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10
 /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10
 /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/backward
 /usr/local/include
 /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/lib/clang/12.0.0/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

Build Configuration:
BB_VERSION           = "1.50.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "x86_64-eko-linux"
MACHINE              = "MY_MACHINE"
DISTRO               = "eko"
DISTRO_VERSION       = "3.0.0"
TUNE_FEATURES        = "m64 zen2"
TARGET_FPU           = ""

I was wondering why clang++ isn't in the same directory as the g++ binary.

I even added nativesdk-clang to our meta-toolchain recipe. This is on hardknott.

[Lorac]

Now using kirkstone. We get a different error.

/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/x86_64-eko-linux/x86_64-eko-linux-clang++ --sysroot=/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux -Og -g -feliminate-unused-debug-types  -pipe -g  CMakeFiles/cmTC_067ea.dir/Debug/testCXXCompiler.cxx.o -o Debug/cmTC_067ea   && :
    /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/x86_64-eko-linux/x86_64-eko-linux-ld: cannot find /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/lib/libm.so.6 inside /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux
    /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/usr/bin/x86_64-eko-linux/x86_64-eko-linux-ld: cannot find /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux/lib/libmvec.so.1 inside /builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux

Looks like the paths are incorrect and something is going on with --sysroot flag. We are trying to build with the native part of the SDK.

I don't know why it is not searching for "lib/libm.so.6" inside "/builds/eko/MY_MACHINE/sysroots/x86_64-ekosdk-linux" instead of the full path? If I don't use sysroot it tries to look inside host library instead and can't find anything.