juicedata
clang compile error with juicefs filesystem
What happened:
Try setup a build system with juicefs.
The clang build works fine when fs is ext4, nfs. but failed with juicefs.
Build command:
/usr/local/llvm/bin/clang++ -v --target=x86_64-linux-gnu
--sysroot=/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc
--gcc-toolchain=/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64
-std=c++20 -fdiagnostics-color=always -fsized-deallocation -pipe -mssse3 -msse4.1 -msse4.2
-mpclmul -mpopcnt -fasynchronous-unwind-tables -ffile-prefix-map=/home/jenkins/build/src/pontus=.
-falign-functions=32 -mbranches-within-32B-boundaries -Wno-error=reserved-identifier
-Wno-error=suggest-destructor-override -fdiagnostics-absolute-paths -fstrict-vtable-pointers
-fexperimental-new-pass-manager -w -O3 -DNDEBUG
--gcc-toolchain=/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64
--ld-path=/usr/bin/ld.gold -rdynamic -no-pie -Wl,-no-pie
-Xlinker --no-undefined contrib/protobuf-cmake/CMakeFiles/protoc.dir/__/protobuf/src/google/protobuf/compiler/main.cc.o
-o contrib/protobuf-cmake/protoc contrib/protobuf-cmake/lib_libprotoc.a contrib/protobuf-cmake/lib_libprotobuf.a
-lpthread contrib/zlib-ng-cmake/lib_zlib.a -Wl,--start-group base/glibc-compatibility/libglibc-compatibility.a
base/glibc-compatibility/memcpy/libmemcpy.a contrib/libcxx-cmake/libcxx.a contrib/libcxxabi-cmake/libcxxabi.a
contrib/libunwind-cmake/libunwind.a -Wl,--end-group
-nodefaultlibs /usr/local/llvm/lib/clang/15.0.7/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a -lc -lm -lrt -lpthread -ldl
Console output: clang version 15.0.7 Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /usr/local/llvm/bin Found candidate GCC installation: /home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/lib/gcc/x86_64-linux-gnu/9 Candidate multilib: .;@m64 Selected multilib: .;@m64 "/usr/bin/ld.gold" --sysroot=/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc -export-dynamic --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -export-dynamic -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o contrib/protobuf-cmake/protoc /home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/usr/lib/../lib64/crt1.o /home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/usr/lib/../lib64/crti.o /home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/lib/gcc/x86_64-linux-gnu/9/crtbegin.o -L/usr/local/llvm/bin/../lib/x86_64-unknown-linux-gnu -L/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/lib/gcc/x86_64-linux-gnu/9 -L/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/lib/../lib64 -L/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/usr/lib/../lib64 -L/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/lib -L/home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/usr/lib -no-pie --no-undefined contrib/protobuf-cmake/CMakeFiles/protoc.dir/__/protobuf/src/google/protobuf/compiler/main.cc.o contrib/protobuf-cmake/lib_libprotoc.a contrib/protobuf-cmake/lib_libprotobuf.a -lpthread contrib/zlib-ng-cmake/lib_zlib.a --start-group base/glibc-compatibility/libglibc-compatibility.a base/glibc-compatibility/memcpy/libmemcpy.a contrib/libcxx-cmake/libcxx.a contrib/libcxxabi-cmake/libcxxabi.a contrib/libunwind-cmake/libunwind.a --end-group /usr/local/llvm/lib/clang/15.0.7/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a -lc -lm -lrt -lpthread -ldl /home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/lib/gcc/x86_64-linux-gnu/9/crtend.o /home/jenkins/build/src/pontus/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/usr/lib/../lib64/crtn.o clang-15: error: unable to execute command: Bus error (core dumped) clang-15: error: linker command failed due to signal (use -v to see invocation)
strace file: upload
core file: upload strace.txt Uploading core.zip…
What you expected to happen: Compile successful.
How to reproduce it (as minimally and precisely as possible): Not sure if you can successful reproduce it. Just upload clang compile strace file and core file.
Anything else we need to know?
Environment:
-
JuiceFS version ： juicefs version 1.1.2+2024-02-04.8dbd89a
-
OS (e.g
cat /etc/os-release): NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" -
Kernel (e.g.
uname -a): Linux 0f07832ca6cc 6.1.0-17-amd64 （CentOS7 Docker in debian) -
Object storage (cloud provider and region, or self maintained): minio
-
Metadata engine info (version, cloud provider managed or self maintained): redis
-
Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage):
-
Others:
Can you reproduce this failure by compiling a open source project? Then we can reproduce it and analyze the root cause.
Reproduce it is not very easy and quickly. But I'll paste all steps here.
- Setup Minio, Redis, JuiceFs ....。 Mount JuiceFs to /data/test
- Setup a docker for build env.
# image name: buildenv
FROM centos:centos7.9.2009
MAINTAINER [email protected]
# Install os packages with local repo
RUN cd /etc/yum.repos.d/ && rename .repo .repo.bak *
COPY CentOS-Base.repo /etc/yum.repos.d/
RUN yum clean all
RUN yum makecache
RUN yum install -y wget openssl openssl-devel python3 sshpass sudo nfs-utils rsync
RUN yum install -y centos-release-scl
RUN yum install -y devtoolset-11*
RUN source /opt/rh/devtoolset-11/enable
# Install c++ compiler
RUN mv /usr/bin/gcc /usr/bin/gcc-4.8.5
RUN mv /usr/bin/g++ /usr/bin/g++-4.8.5
RUN mv /usr/bin/c++ /usr/bin/c++-4.8.5
RUN ln -s /opt/rh/devtoolset-11/root/bin/gcc /usr/bin/gcc
RUN ln -s /opt/rh/devtoolset-11/root/bin/g++ /usr/bin/g++
RUN ln -s /opt/rh/devtoolset-11/root/bin/c++ /usr/bin/c++
RUN ln -s /opt/rh/devtoolset-11/root/lib/gcc/x86_64-redhat-linux/11 /usr/lib/gcc/x86_64-redhat-linux/11
ADD jdk1.8.0_291.tar.gz /opt/
ADD apache-maven-3.6.3.tar.gz /opt/
ADD git-2.29.2.tar.gz /opt
ADD cmake-3.16.6-Linux-x86_64.tar.gz /usr/local/
RUN rm -f /usr/bin/git
RUN alternatives --install /usr/bin/java java /opt/jdk1.8.0_291/bin/java 500
RUN alternatives --install /usr/bin/mvn mvn /opt/apache-maven-3.6.3/bin/mvn 500
RUN alternatives --install /usr/bin/git git /opt/git-2.29.2/bin/git 500
RUN alternatives --install /usr/bin/cmake cmake /usr/local/cmake-3.16.6-Linux-x86_64/bin/cmake 500
ADD ninja-1.10.1.tar.gz /tmp
RUN cd /tmp/ninja-1.10.1 && cmake -Bbuild-cmake -H. && cmake --build build-cmake && cp build-cmake/ninja /usr/local/bin/ninja
RUN ninja --version
RUN rm -f -r /tmp/ninja-1.10.1
ADD llvm-project-15.0.7.src.tar.gz /tmp
RUN cd /tmp/llvm-project-15.0.7.src/ && cmake -S llvm -B build_release -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/llvm -G "Ninja" -DLLVM_USE_LINKER="gold" -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;lldb" -DLLVM_ENABLE_RUNTIMES="libc;compiler-rt;libcxx;libcxxabi;libunwind" -DCMAKE_C_COMPILER=/opt/rh/devtoolset-11/root/usr/bin/gcc -DCMAKE_CXX_COMPILER=/opt/rh/devtoolset-11/root/usr/bin/g++
RUN cd /tmp/llvm-project-15.0.7.src/build_release/ && ninja -j 12 && ninja install
RUN rm -f -r /tmp/llvm-project-15.0.7.src
RUN /usr/local/llvm/bin/clang --version
ENV JAVA_HOME /opt/jdk1.8.0_291
ENV MAVEN_HOME /opt/apache-maven-3.6.3
ENV PATH $PATH:$JAVA_HOME/bin
ENV MAVEN_CUSTOM_LOCAL /home/jenkins
- Compile clickhouse with docker
# 打包构建到临时目录
SNAPSHOT_TMPDIR=/data/test
docker run \
-v /home/jenkins/build:/home/jenkins/script \
-v ${SNAPSHOT_TMPDIR}:/home/jenkins/build/src \
--privileged \
<your build docker> \
sh /home/jenkins/script/build.sh
build.sh:
cd /home/jenkins/build/src/
git clone --branch v22.8.4.7-lts https://github.com/ClickHouse/ClickHouse.git
cd ClickHouse
git submodule update --init --recursive --jobs 6
WORKSPACE=$(git rev-parse --show-toplevel)
echo 代码工作空间[${WORKSPACE}].
cd ${WORKSPACE}
echo 清理之前的构建目录...
BUILD_DIR=${BUILD_DIR:-$WORKSPACE/build}
rm -rf ${BUILD_DIR}
mkdir -p ${BUILD_DIR}
echo 更新SubModule...
git submodule update --init --recursive --jobs 6
echo 开始构建代码...
cd ${BUILD_DIR}
cmake -DENABLE_TESTS=OFF -DBENCHMARK_ENABLE_TESTING=OFF -DENABLE_S3=OFF \
-DENABLE_CLICKHOUSE_ALL=OFF \
-DENABLE_CLICKHOUSE_SERVER=ON -DENABLE_CLICKHOUSE_CLIENT=ON \
-DENABLE_CLICKHOUSE_PONTUS_TOOL=ON \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=/usr/local/llvm/bin/clang \
-DCMAKE_CXX_COMPILER=/usr/local/llvm/bin/clang++ \
-DENABLE_CCACHE=0 ${WORKSPACE}
ninja
maybe upload docker image and clickhouse source code to "百度网盘" can help you? I'll upload them later.
