whisper.cpp Questions about building CLBlast for android

I've tried the instructions in https://github.com/ggerganov/whisper.cpp/tree/ac283dbce7d42735e3ed985329037bf23fe180aa/examples/whisper.android

# In path/to/CLBlast (we assume OpenCL-Headers relative location)
$ANDROID_SDK_PATH/cmake/3.22.1/bin/cmake .. \
    -DCMAKE_SYSTEM_NAME=Android \
    -DCMAKE_SYSTEM_VERSION=33 \
    -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
    -DCMAKE_ANDROID_NDK=$ANDROID_NDK_PATH \
    -DCMAKE_ANDROID_STL_TYPE=c++_static \
    -DOPENCL_ROOT=$(readlink -f ../../OpenCL-Headers) \
    -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \
    -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH

# Build libclblast.so
make -j4

But I was received error during make -j4 :

ld.lld: error: /usr/local/cuda/lib64/libOpenCL.so is incompatible with aarch64linux

I have no idea why it's trying to reference the cuda openCL library. I have OpenCL-headers in the correct relative path. I ran the command in CLBlast/build.

Does anyone have any guidance on how to build properly?

Apr 02 '24 02:04 liam-mceneaney

The makefile try to find libOpenCL.so from OPENCL_ROOT and your system library folders. You should copy libOpenCL.so from android device via adb shell, and put that in OpenCL-Headers folder. Then makefile will find libOpenCL.so file correctly. I spent 2 weeks for build android app with CLBlast and succeed, so if you have any question about it please ask me freely.

Apr 04 '24 01:04 colonist4

Ah thank you so much! I was so confused on the order of operations on that. I'll update when I succeed!

Apr 05 '24 16:04 liam-mceneaney

@colonist4 I am still a bit lost, for the paths to CLBlast, do I include the build folder?

GGML_CLBLAST=ON
CLBLAST_HOME=/path/to/CLBlast
OPENCL_LIB=/path/to/libOpenCL.so
OPENCL_ROOT=/path/to/OpenCL-Headers

So should it be /path/to/CLBLast/build? Is /path/to/libOpenCL.so this path? lib/src/main/jniLibs/arm64-v8a/libOpenCL.so

Also in this part:

mkdir lib/src/main/jniLibs/arm64-v8a
adb pull /system/vendor/lib64/egl/libGLES_mali.so lib/src/main/jniLibs/arm64-v8a/libOpenCL.so

I have a libGLESv2_adreno.so Do I just replace libGLES_mali with that instead? I don't really understand what exactly is the "make" command generating and why.

Would you be able to share how yours is structured so I can get an idea of what looks right?

Apr 11 '24 15:04 liam-mceneaney

Hi, I'm facing similar issue:

I pulled libOpenCL.so directly (@liam-mceneaney I also had adreno but then building the clblast failed, (https://github.com/CNugteren/CLBlast/issues/537)).

adb pull /system/vendor/lib64/libOpenCL.so /path/to/libOpenCL.so

~~Now I'm playing around with gradle.properties but cannot figure where to include clblast build folder ...~~ edit: the cmakelists includes build/ folder from clblast root.

Currently I'm using:

GGML_HOME=/home/samoh/Documents/GitHub/LocalLM/grafa/ggml
GGML_CLBLAST=ON
CLBLAST_HOME=/home/samoh/Documents/GitHub/LocalLM/grafa/CLBlast
OPENCL_LIB=/home/samoh/Documents/GitHub/LocalLM/lib/whispercpp/src/main/jniLibs/arm64-v8a/libOpenCL.so
OPENCL_ROOT=/home/samoh/Documents/GitHub/LocalLM/grafa/OpenCL-Headers

When I try to build the project, these are the errors: (I also had to remove --copy-dt-needed-entries, as it said ld: error: unknown argument '--copy-dt-needed-entries' otherwise)

Logs

...
C/C++: /home/samoh/Documents/GitHub/LocalLM/lib/whispercpp/.cxx/Release/2i2o436m/arm64-v8a/_deps/whisper.cpp-repo-src/whisper.cpp:202:29: warning: unused function 'ggml_mul_mat_pad' [-Wunused-function]
C/C++: static struct ggml_tensor * ggml_mul_mat_pad(struct ggml_context * ctx, struct ggml_tensor * x, struct ggml_tensor * y, int pad = 32) {
C/C++:                             ^
C/C++: 2 warnings generated.
C/C++: : && /home/samoh/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=aarch64-none-linux-android26 --sysroot=/home/samoh/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security   -O3 -DNDEBUG  -static-libstdc++ -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments  -Wl,--gc-sections   -Wl,--gc-sections -Wl,--exclude-libs,ALL -flto -shared -Wl,-soname,libwhisper_v8fp16_va.so -o /home/samoh/Documents/GitHub/LocalLM/lib/whispercpp/build/intermediates/cxx/Release/2i2o436m/obj/arm64-v8a/libwhisper_v8fp16_va.so CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o CMakeFiles/whisper_v8fp16_va.dir/jni.c.o  /home/samoh/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/26/liblog.so  -landroid  /home/samoh/Documents/GitHub/LocalLM/lib/whispercpp/build/intermediates/cxx/Release/2i2o436m/obj/arm64-v8a/libggml.so  /home/samoh/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/26/libm.so  /home/samoh/Documents/GitHub/LocalLM/grafa/CLBlast/build/libclblast.so  /home/samoh/Documents/GitHub/LocalLM/lib/whispercpp/src/main/jniLibs/arm64-v8a/libOpenCL.so  -pthread  -ldl  -latomic -lm && :
C/C++: ld: error: undefined symbol: ggml_allocr_max_size
C/C++: >>> referenced by whisper.cpp:482 (_deps/whisper.cpp-repo-src/whisper.cpp:482)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced by whisper.cpp:482 (_deps/whisper.cpp-repo-src/whisper.cpp:482)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced by whisper.cpp:482 (_deps/whisper.cpp-repo-src/whisper.cpp:482)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced 5 more times
C/C++: ld: error: undefined symbol: ggml_allocr_free
C/C++: >>> referenced by whisper.cpp:508 (_deps/whisper.cpp-repo-src/whisper.cpp:508)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced by whisper.cpp:508 (_deps/whisper.cpp-repo-src/whisper.cpp:508)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced by whisper.cpp:508 (_deps/whisper.cpp-repo-src/whisper.cpp:508)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced 7 more times
C/C++: ld: error: undefined symbol: ggml_allocr_new_from_buffer
C/C++: >>> referenced by whisper.cpp:511 (_deps/whisper.cpp-repo-src/whisper.cpp:511)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced by whisper.cpp:511 (_deps/whisper.cpp-repo-src/whisper.cpp:511)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced by whisper.cpp:511 (_deps/whisper.cpp-repo-src/whisper.cpp:511)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_init_state)
C/C++: >>> referenced 3 more times
C/C++: ld: error: undefined symbol: ggml_allocr_alloc
C/C++: >>> referenced by whisper.cpp:921 (_deps/whisper.cpp-repo-src/whisper.cpp:921)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(kv_cache_init(whisper_hparams const&, whisper_kv_cache&, ggml_backend*, ggml_type, int))
C/C++: >>> referenced by whisper.cpp:922 (_deps/whisper.cpp-repo-src/whisper.cpp:922)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(kv_cache_init(whisper_hparams const&, whisper_kv_cache&, ggml_backend*, ggml_type, int))
C/C++: >>> referenced by whisper.cpp:1534 (_deps/whisper.cpp-repo-src/whisper.cpp:1534)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_model_load(whisper_model_loader*, whisper_context&))
C/C++: >>> referenced 4 more times
C/C++: ld: error: undefined symbol: ggml_allocr_new_measure_from_backend
C/C++: >>> referenced by whisper.cpp:490 (_deps/whisper.cpp-repo-src/whisper.cpp:490)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_allocr_graph_init(whisper_allocr&, ggml_backend*, std::__ndk1::function<ggml_cgraph* ()>&&))
C/C++: ld: error: undefined symbol: ggml_allocr_alloc_graph
C/C++: >>> referenced by whisper.cpp:494 (_deps/whisper.cpp-repo-src/whisper.cpp:494)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_allocr_graph_init(whisper_allocr&, ggml_backend*, std::__ndk1::function<ggml_cgraph* ()>&&))
C/C++: >>> referenced by whisper.cpp:2103 (_deps/whisper.cpp-repo-src/whisper.cpp:2103)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_encode_internal(whisper_context&, whisper_state&, int, int, bool (*)(void*), void*))
C/C++: >>> referenced by whisper.cpp:2120 (_deps/whisper.cpp-repo-src/whisper.cpp:2120)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_encode_internal(whisper_context&, whisper_state&, int, int, bool (*)(void*), void*))
C/C++: >>> referenced 2 more times
C/C++: ld: error: undefined symbol: ggml_allocr_reset
C/C++: >>> referenced by whisper.cpp:2099 (_deps/whisper.cpp-repo-src/whisper.cpp:2099)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_encode_internal(whisper_context&, whisper_state&, int, int, bool (*)(void*), void*))
C/C++: >>> referenced by whisper.cpp:2116 (_deps/whisper.cpp-repo-src/whisper.cpp:2116)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_encode_internal(whisper_context&, whisper_state&, int, int, bool (*)(void*), void*))
C/C++: >>> referenced by whisper.cpp:2131 (_deps/whisper.cpp-repo-src/whisper.cpp:2131)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_encode_internal(whisper_context&, whisper_state&, int, int, bool (*)(void*), void*))
C/C++: >>> referenced 1 more times
C/C++: ld: error: undefined symbol: ggml_cpu_has_cublas
C/C++: >>> referenced by whisper.cpp:3868 (_deps/whisper.cpp-repo-src/whisper.cpp:3868)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_print_system_info)
C/C++: ld: error: undefined symbol: ggml_allocr_is_measure
C/C++: >>> referenced by whisper.cpp:1689 (_deps/whisper.cpp-repo-src/whisper.cpp:1689)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_build_graph_conv(whisper_context&, whisper_state&, int))
C/C++: >>> referenced by whisper.cpp:2169 (_deps/whisper.cpp-repo-src/whisper.cpp:2169)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_build_graph_decoder(whisper_context&, whisper_state&, whisper_batch const&))
C/C++: >>> referenced by whisper.cpp:2170 (_deps/whisper.cpp-repo-src/whisper.cpp:2170)
C/C++: >>>               CMakeFiles/whisper_v8fp16_va.dir/_deps/whisper.cpp-repo-src/whisper.cpp.o:(whisper_build_graph_decoder(whisper_context&, whisper_state&, whisper_batch const&))
C/C++: >>> referenced 3 more times
C/C++: clang++: error: linker command failed with exit code 1 (use -v to see invocation)

Any ideas what I'm doing wrong?

Apr 12 '24 08:04 samolego

@liam-mceneaney The build instructions in android example readme contains both "CLBlast.so build" and "android application build". So may it makes you confused.

This

GGML_CLBLAST=ON
CLBLAST_HOME=/path/to/CLBlast
OPENCL_LIB=/path/to/libOpenCL.so
OPENCL_ROOT=/path/to/OpenCL-Headers

And this

mkdir lib/src/main/jniLibs/arm64-v8a
adb pull /system/vendor/lib64/egl/libGLES_mali.so lib/src/main/jniLibs/arm64-v8a/libOpenCL.so

are about "android application build". And let's put it off for a while.

Firs,t you have to "build CLBlast.so". I will describe build steps in detail.

In your working directory (/working/directory)
clone CLBlast repository (git clone https://github.com/CNugteren/CLBlast.git)
clone OpenCL-headers repository (git clone https://github.com/KhronosGroup/OpenCL-Headers.git)
In OpenCL-Headers dir, pull libOpenCL.so from your android device. (adb pull /system/vendor/lib64/egl/libGLESv2_adreno.so ./libOpenCL.so)
In CLBlast dir, make 'build' dir (cd CLBlast && mkdir build)
In build dir, run cmake command (should set ANDROID_NDK_PATH, ANDROID_NDK_PATH env)

$ANDROID_SDK_PATH/cmake/3.22.1/bin/cmake .. \
    -DCMAKE_SYSTEM_NAME=Android \
    -DCMAKE_SYSTEM_VERSION=33 \
    -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
    -DCMAKE_ANDROID_NDK=$ANDROID_NDK_PATH \
    -DCMAKE_ANDROID_STL_TYPE=c++_static \
    -DOPENCL_ROOT=../../OpenCL-Headers \
    -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \
    -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH

run make command (make -j4)

Then you will get libCLBlast.so file. And use it for further "android application build" steps. If you have any issues while building, ask me anytime. Good luck!

Apr 15 '24 01:04 colonist4

@samolego The android ndk requires using LLVM compiler since NDK r17 (maybe?) so the android studio try to compile all related sources with NDK's LLVM compiler. The main issue is LLVM does not support "--copy-dt-needed-entries" option (https://reviews.llvm.org/D56647). So it failed with the error message you met.

I don't know how to not using LLVM compiler for android project, and don't know alternative options in LLVM. So i just commented out that lines in Makefile, and it works. Probably that option is used in GGML project, you can find that in src/CMakeLists.txt (https://github.com/ggerganov/ggml/blob/98875cdb7e9ceeb726d1c196d2fecb3cbb59b93a/src/CMakeLists.txt#L202). Just comment it out and try to build and see if it works.

Apr 15 '24 01:04 colonist4

@liam-mceneaney I'm not using adreno, so i don't know whether libGLESv2_adreno.so works for building CLBlast. But it seems like @samolego knows whether it is. The comment https://github.com/CNugteren/CLBlast/issues/537#issuecomment-2051184157 may help you, please refer to.

Apr 15 '24 01:04 colonist4

Just comment it out and try to build and see if it works.

Have done it, the logs above were with option removed :| ... I'll look again in the setup, but not sure what I've missed.

Apr 15 '24 04:04 samolego

@liam-mceneaney did you perhaps succeed?

Apr 26 '24 07:04 samolego

@liam-mceneaney I'm not using adreno, so i don't know whether libGLESv2_adreno.so works for building CLBlast. But it seems like @samolego knows whether it is. The comment CNugteren/CLBlast#537 (comment) may help you, please refer to.

i have a pixel 2xl . i could not find a opencl.so on it. but it has a snapdragon gpu an adreno 540. So i found on. a repo libopencl.so . i built clblast with it. defind the gradle.properites but the app crashes on launch

GGML_CLBLAST=ON CLBLAST_HOME=/Users/jamshaidali/Development/Codebases/staging/whisper.cpp-master/test/CLBlast/build/libclblast.so OPENCL_LIB=/Users/jamshaidali/Development/Codebases/staging/whisper.cpp-master/examples/whisper.android/lib/src/main/jniLibs/arm64-v8a/libOpenCL.so OPENCL_ROOT=/Users/jamshaidali/Development/Codebases/staging/whisper.cpp-master/test/OpenCL-Headers

Aug 19 '24 23:08 stopthinking102

whisper.cpp whisper.cpp copied to clipboard

Questions about building CLBlast for android

whisper.cpp
whisper.cpp copied to clipboard