Compile bug: Fails to compile with undefined references in libggml.so

[sjwhitak]

Git commit

Tag b4607

Operating systems

Linux

GGML backends

CPU

Problem description & steps to reproduce

I'm using RHEL8.10. cmake 3.26.5 make 4.2.1 gmake 4.2.1.

I am trying to compile with no compile flags to execute on the CPU. This used to work when running make to build llama.cpp, but I am not smart enough to find which commit that occurred at before running a git pull to update llama.cpp, sorry. Please let me know of other logs required. When executing cd build/; make -d, and ltrace and strace, I wasn't able find additional info that seemed useful, but I can dump that if need be.

First Bad Commit

No response

Compile command

cmake -B build -DGGML_CCACHE=OFF
cmake --build build --config Release

Relevant log output

cmake -B build -DGGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Configuring done (0.8s)
-- Generating done (9.8s)
-- Build files have been written to: /home/steven/code/llama.cpp/build




cmake --build build --config Release
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Configuring done (0.8s)
-- Generating done (10.7s)
-- Build files have been written to: /home/steven/code/llama.cpp/build
~/projects/LLM/llama.cpp$ cmake --build build --config Release
[  4%] Built target ggml-base
[  9%] Built target ggml-cpu
[ 10%] Built target ggml
[ 11%] Building CXX object src/CMakeFiles/llama.dir/llama.cpp.o
[ 11%] Building CXX object src/CMakeFiles/llama.dir/llama-adapter.cpp.o
[ 11%] Building CXX object src/CMakeFiles/llama.dir/llama-batch.cpp.o
[ 12%] Building CXX object src/CMakeFiles/llama.dir/llama-chat.cpp.o
[ 12%] Building CXX object src/CMakeFiles/llama.dir/llama-context.cpp.o
[ 13%] Building CXX object src/CMakeFiles/llama.dir/llama-grammar.cpp.o
[ 13%] Building CXX object src/CMakeFiles/llama.dir/llama-hparams.cpp.o
[ 14%] Building CXX object src/CMakeFiles/llama.dir/llama-impl.cpp.o
[ 14%] Building CXX object src/CMakeFiles/llama.dir/llama-kv-cache.cpp.o
[ 14%] Building CXX object src/CMakeFiles/llama.dir/llama-model-loader.cpp.o
[ 15%] Building CXX object src/CMakeFiles/llama.dir/llama-model.cpp.o
[ 15%] Building CXX object src/CMakeFiles/llama.dir/llama-quant.cpp.o
[ 16%] Building CXX object src/CMakeFiles/llama.dir/llama-sampling.cpp.o
[ 16%] Building CXX object src/CMakeFiles/llama.dir/llama-vocab.cpp.o
[ 17%] Linking CXX shared library ../bin/libllama.so
[ 20%] Built target llama
[ 20%] Generating build details from Git
-- Found Git: /usr/bin/git (found version "2.43.5") 
[ 20%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[ 20%] Built target build_info
[ 21%] Building CXX object common/CMakeFiles/common.dir/arg.cpp.o
[ 21%] Building CXX object common/CMakeFiles/common.dir/chat.cpp.o
[ 22%] Building CXX object common/CMakeFiles/common.dir/common.cpp.o
[ 23%] Building CXX object common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o
[ 23%] Building CXX object common/CMakeFiles/common.dir/log.cpp.o
[ 24%] Building CXX object common/CMakeFiles/common.dir/ngram-cache.cpp.o
[ 24%] Building CXX object common/CMakeFiles/common.dir/sampling.cpp.o
[ 25%] Building CXX object common/CMakeFiles/common.dir/speculative.cpp.o
[ 25%] Linking CXX static library libcommon.a
[ 25%] Built target common
[ 26%] Building CXX object tests/CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o
[ 26%] Linking CXX executable ../bin/test-tokenizer-0
../bin/libggml.so: undefined reference to `vtable for std::filesystem::__cxx11::filesystem_error'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator*() const'
../bin/libggml.so: undefined reference to `typeinfo for std::filesystem::__cxx11::filesystem_error'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_find_extension() const'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::~filesystem_error()'
../bin/libggml.so: undefined reference to `std::filesystem::status(std::filesystem::__cxx11::path const&)'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator++()'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::directory_iterator(std::filesystem::__cxx11::path const&, std::filesystem::directory_options, std::error_code*)'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::_M_gen_what()'
collect2: error: ld returned 1 exit status
gmake[2]: *** [tests/CMakeFiles/test-tokenizer-0.dir/build.make:102: bin/test-tokenizer-0] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1856: tests/CMakeFiles/test-tokenizer-0.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2





ldd build/bin/libggml.so 
	linux-vdso.so.1 (0x00007ffe9599d000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f218e3b4000)
	libggml-cpu.so => /home/steven/code/llama.cpp/build/bin/libggml-cpu.so (0x00007f218e0e7000)
	libggml-base.so => /home/steven/code/llama.cpp/build/bin/libggml-base.so (0x00007f218de11000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f218da7c000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f218d6fa000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f218d4e2000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f218d10c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f218e7c8000)
	libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f218ced4000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f218ccb4000)

[lexasub]

@sjwhitak what version of gcc do you use? (or clang, or other compiler)

[lexasub]

@sjwhitak try cmake -B build -DGGML_CCACHE=OFF -DCMAKE_CXX_STANDARD=17 RHEL8.10 have old gcc, with other default standart

[saraph]

@lexasub I'm running into the same error on gcc 8.5.0:

$ gcc --version
gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I tried your suggested solution of

cmake -B build -DGGML_CCACHE=OFF -DCMAKE_CXX_STANDARD=17

but the error persists.

Please let me know if you have other suggestions. Thanks.

[lexasub]

@saraph you may try add set(CMAKE_CXX_STANDARD 17) in CMakeLists.txt file. or you may try up gcc))

[saraph]

Thanks @lexasub. I updated gcc to 13.2 (they don't yet have 14.2 on my host) and added to CMakeLists.txt the line you suggested above:

$ gcc --version
gcc (Spack GCC) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

When I tried to build with gcc 13.2, I got the same error:

$ cmake --build build --config Release -j 8
[  0%] Built target build_info
[  0%] Built target sha1
[  1%] Built target sha256
[  2%] Built target xxhash
[  6%] Built target ggml-base
[ 10%] Built target ggml-cpu
[ 11%] Built target ggml
[ 11%] Linking CXX executable ../../bin/llama-gguf-hash
[ 11%] Linking CXX executable ../../bin/llama-gguf
[ 21%] Built target llama
[ 21%] Linking C executable ../bin/test-c
[ 22%] Linking CXX executable ../../bin/llama-simple
[ 23%] Linking CXX executable ../../bin/llama-simple-chat
[ 25%] Built target llava
[ 25%] Linking CXX executable ../../bin/llama-quantize-stats
../../bin/libggml.so: undefined reference to `vtable for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator*() const'
../../bin/libggml.so: undefined reference to `typeinfo for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_find_extension() const'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::~filesystem_error()'
../../bin/libggml.so: undefined reference to `std::filesystem::status(std::filesystem::__cxx11::path const&)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator++()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::directory_iterator(std::filesystem::__cxx11::path const&, std::filesystem::directory_options, std::error_code*)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::_M_gen_what()'
collect2: error: ld returned 1 exit status
../../bin/libggml.so: undefined reference to `vtable for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator*() const'
../../bin/libggml.so: undefined reference to `typeinfo for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_find_extension() const'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::~filesystem_error()'
../../bin/libggml.so: undefined reference to `std::filesystem::status(std::filesystem::__cxx11::path const&)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator++()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::directory_iterator(std::filesystem::__cxx11::path const&, std::filesystem::directory_options, std::error_code*)'
../../bingmake[2]: *** [examples/gguf-hash/CMakeFiles/llama-gguf-hash.dir/build.make:106: bin/llama-gguf-hash] Error 1
/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::_M_gen_what()'
collect2: error: ld returned 1 exit status
gmake[1]: *** [CMakeFiles/Makefile2:2662: examples/gguf-hash/CMakeFiles/llama-gguf-hash.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
gmake[2]: *** [examples/gguf/CMakeFiles/llama-gguf.dir/build.make:100: bin/llama-gguf] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2799: examples/gguf/CMakeFiles/llama-gguf.dir/all] Error 2
[ 31%] Built target common
[ 32%] Linking CXX static library libllava_static.a
../bin/libggml.so: undefined reference to `vtable for std::filesystem::__cxx11::filesystem_error'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator*() const'
../bin/libggml.so: undefined reference to `typeinfo for std::filesystem::__cxx11::filesystem_error'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_find_extension() const'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::~filesystem_error()'
../bin/libggml.so: undefined reference to `std::filesystem::status(std::filesystem::__cxx11::path const&)'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator++()'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::directory_iterator(std::filesystem::__cxx11::path const&, std::filesystem::directory_options, std::error_code*)'
../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::_M_gen_what()'
collect2: error: ld returned 1 exit status
gmake[2]: *** [tests/CMakeFiles/test-c.dir/build.make:101: bin/test-c] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2476: tests/CMakeFiles/test-c.dir/all] Error 2
../../bin/libggml.so: undefined reference to `vtable for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator*() const'
../../bin/libggml.so: undefined reference to `typeinfo for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_find_extension() const'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::~filesystem_error()'
../../bin/libggml.so: undefined reference to `std::filesystem::status(std::filesystem::__cxx11::path const&)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator++()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::directory_iterator(std::filesystem::__cxx11::path const&, std::filesystem::directory_options, std::error_code*)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::_M_gen_what()'
collect2: error: ld returned 1 exit status
../../bin/libggml.so: undefined reference to `vtable for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator*() const'
../../bin/libggml.so: undefined reference to `typeinfo for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_find_extension() const'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::~filesystem_error()'
../../bin/libggml.so: undefined reference to `std::filesystem::status(std::filesystem::__cxx11::path const&)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator++()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::directory_iterator(std::filesystem::__cxx11::path const&, std::filesystem::directory_options, std::error_code*)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::_M_gen_what()'
collect2: error: ld returned 1 exit status
gmake[2]: *** [examples/simple/CMakeFiles/llama-simple.dir/build.make:101: bin/llama-simple] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:3386: examples/simple/CMakeFiles/llama-simple.dir/all] Error 2
gmake[2]: *** [examples/simple-chat/CMakeFiles/llama-simple-chat.dir/build.make:101: bin/llama-simple-chat] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:3415: examples/simple-chat/CMakeFiles/llama-simple-chat.dir/all] Error 2
../../bin/libggml.so: undefined reference to `vtable for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator*() const'
../../bin/libggml.so: undefined reference to `typeinfo for std::filesystem::__cxx11::filesystem_error'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_find_extension() const'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::~filesystem_error()'
../../bin/libggml.so: undefined reference to `std::filesystem::status(std::filesystem::__cxx11::path const&)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::operator++()'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::directory_iterator::directory_iterator(std::filesystem::__cxx11::path const&, std::filesystem::directory_options, std::error_code*)'
../../bin/libggml.so: undefined reference to `std::filesystem::__cxx11::filesystem_error::_M_gen_what()'
collect2: error: ld returned 1 exit status
gmake[2]: *** [examples/quantize-stats/CMakeFiles/llama-quantize-stats.dir/build.make:103: bin/llama-quantize-stats] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:3693: examples/quantize-stats/CMakeFiles/llama-quantize-stats.dir/all] Error 2
[ 32%] Built target llava_static
gmake: *** [Makefile:146: all] Error 2

[saraph]

Interestingly, I was successfully able to install llama-cpp-python from pip even though according to the docs it also uses gcc to compile llama.cpp:

$ pip install llama-cpp-python
Collecting llama-cpp-python
  Downloading llama_cpp_python-0.1.10.tar.gz (518 kB)
     |████████████████████████████████| 518 kB 9.3 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... done
  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.10-cp36-cp36m-linux_x86_64.whl size=178025 sha256=6114daf292710a8797ca88c59327215f6af102d0cb2a3f4edaa543a70623d446
  Stored in directory: /gpfs/home/ds2827/.cache/pip/wheels/ed/1a/c5/f7b111263b76f280d871ccd1fb981ce0c338700c7840903f02
Successfully built llama-cpp-python
Installing collected packages: llama-cpp-python
Successfully installed llama-cpp-python-0.1.10

Where on my system should it have built/installed llama.cpp? I see the following in my site-packages/llama_cpp dir within the pip venv:

$ ls -latr
total 450
-rw-rw----  1 ds2827 ds2827     46 Feb 10 16:34 __init__.py
-rwxrwx--x  1 ds2827 ds2827 421800 Feb 10 16:34 libllama.so
-rw-rw----  1 ds2827 ds2827  11210 Feb 10 16:34 llama.py
-rw-rw----  1 ds2827 ds2827   7128 Feb 10 16:34 llama_cpp.py
drwxr-xr-- 60 ds2827 ds2827   4096 Feb 10 16:34 ..
drwxrwx---  3 ds2827 ds2827   4096 Feb 10 16:34 .
drwxrwx---  2 ds2827 ds2827   4096 Feb 10 16:34 __pycache__

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[Thireus]

Alternative solution: https://github.com/ggml-org/llama.cpp/issues/10978#issuecomment-2597511343