llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

blas build: use configuration from pkg_check_modules(DepBLAS openblas) and alike

Open khumarahn opened this issue 1 year ago • 4 comments

Hi. On my system (a gentoo), cmake build process with OpenBLAS or BLIS fails because linker options like -openblas or -lblis are missing.

With BLIS:

$ cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=FLAME -DBUILD_NUMBER=1 && cmake --build build --config Release -v
...
[ 26%] Linking CXX executable ../bin/test-tokenizer-0
cd llama.cpp/build/tests && /usr/bin/cmake -E cmake_link_script CMakeFiles/test-tokenizer-0.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o" -o ../bin/test-tokenizer-0  -Wl,-rpath,llama.cpp/build/bin: ../common/libcommon.a ../bin/libllama.so ../bin/libggml.so ../bin/libggml-cpu.so ../bin/libggml-blas.so ../bin/libggml-base.so
/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/bin/ld: ../bin/libggml-blas.so: undefined reference to `bli_thread_set_num_threads'
/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/bin/ld: ../bin/libggml-blas.so: undefined reference to `cblas_sgemm'
collect2: error: ld returned 1 exit status
gmake[2]: *** [tests/CMakeFiles/test-tokenizer-0.dir/build.make:103: bin/test-tokenizer-0] Error 1
gmake[2]: Leaving directory 'llama.cpp/build'
gmake[1]: *** [CMakeFiles/Makefile2:1906: tests/CMakeFiles/test-tokenizer-0.dir/all] Error 2
gmake[1]: Leaving directory 'llama.cpp/build'
gmake: *** [Makefile:146: all] Error 2

or with OpenBLAS:

$ cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DBUILD_NUMBER=1 && cmake --build build --config Release -v
[ 26%] Linking CXX executable ../bin/test-tokenizer-0
cd llama.cpp/build/tests && /usr/bin/cmake -E cmake_link_script CMakeFiles/test-tokenizer-0.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o" -o ../bin/test-tokenizer-0  -Wl,-rpath,llama.cpp/build/bin: ../common/libcommon.a ../bin/libllama.so ../bin/libggml.so ../bin/libggml-cpu.so ../bin/libggml-blas.so ../bin/libggml-base.so
/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/bin/ld: ../bin/libggml-blas.so: undefined reference to `openblas_set_num_threads'
/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/bin/ld: ../bin/libggml-blas.so: undefined reference to `cblas_sgemm'
collect2: error: ld returned 1 exit status
gmake[2]: *** [tests/CMakeFiles/test-tokenizer-0.dir/build.make:103: bin/test-tokenizer-0] Error 1
gmake[2]: Leaving directory 'llama.cpp/build'
gmake[1]: *** [CMakeFiles/Makefile2:1906: tests/CMakeFiles/test-tokenizer-0.dir/all] Error 2
gmake[1]: Leaving directory 'llama.cpp/build'
gmake: *** [Makefile:146: all] Error 2

The problem seems to be quite basic: the build process is not using DepBLAS_LIBRARIES and DepBLAS_LINKER_FLAGS that come from pkg_check_modules(DepBLAS openblas) or similar. So I added a couple of lines to fix it. I'm not sure that this is the right fix, and I don't know cmake, but it seems sensible.

P.S.: Edited because at first I was wrong P.P.S.: Added the error output so people who have the same problem are more likely to find it

khumarahn avatar Feb 07 '25 19:02 khumarahn

BLAS_LIBRARIES and BLAS_LINKER_FLAGS should already be set by https://cmake.org/cmake/help/latest/module/FindBLAS.html. I believe pkgconfig is used only to find the include directories, since that's something that FindBLAS does not do.

slaren avatar Feb 09 '25 23:02 slaren

It makes sense, but I'm not sure why that isn't working. FindBLAS is pointing me to /usr/lib64/libblas.so which is installed by lapack, neither blis nor openblas.

A slightly modified cmake script to print out the variables gives:

-- BLAS found:
--     Libraries: /usr/lib64/libblas.so
--     Include dirs: 
--     Linker flags: 
-- Found PkgConfig: /usr/bin/pkg-config (found version "2.3.0")
-- Checking for module 'openblas64'
--   Package 'openblas64' not found
-- Checking for module 'openblas'
--   Found openblas, version 0.3.29
-- DepBLAS found:
--     Libraries: openblas
--     Include dirs: /usr/include/openblas
--     Linker flags: 

khumarahn avatar Feb 10 '25 00:02 khumarahn

I think I understand this better now. But I'm lost even more.

To the best of my current understanding, this issue is a combination of two problems:

  • if BLA_PREFER_PKGCONFIG is set, then cmake is looking for blas with pkgconfig without taking into account BLA_VENDOR https://github.com/Kitware/CMake/blob/6a3631207a768e5b8c6b7446763a2085dfb8fdf1/Modules/FindBLAS.cmake#L291 This can be considered as broken, but it is not normally happening because BLA_PREFER_PKGCONFIG in llama.cpp is not set.
  • But Gentoo patches cmake, adding set(BLA_PREFER_PKGCONFIG ON) in the beginning of FindBLAS.cmake. https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-build/cmake/files/cmake-3.27.0_rc1-0003-Prefer-pkgconfig-in-FindBLAS.patch This looks innocent and would work if not for the previous point...

UPD: an old bug in gentoo: https://bugs.gentoo.org/736547

khumarahn avatar Feb 10 '25 11:02 khumarahn

On Ubuntu 24.04, it works as it is for both OpenBLAS and BLIS. Maybe we can add a workaround for distributions where FindBLAS is broken, but it should be optional.

slaren avatar Feb 10 '25 20:02 slaren

We fixed (or rather worked around) this for gentoo https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8471ec49af94d79ebafca2fb9a0b69e0c4d16518 https://gitweb.gentoo.org/repo/proj/guru.git/commit/?id=d233181e96f30991c95ac43d5c70c8a4df145b3b

Since it turned out that the problem is not with llama.cpp in the first place, but rather with cmake and gentoo's patch for cmake, I'm closing this PR.

khumarahn avatar Jun 17 '25 10:06 khumarahn