Trying to compile for hipblas/ROCm with `cmake` causes `ggml.h not found` error

[lyssieth]

I have the latest commit at time of writing (8f253ef3af1c62c04316ba4afa7145fc4d701a8c), and am trying to use cmake to compile it instead of make, because I'm trying to debug an issue with building Rust bindings for it (which uses cmake).

I've been able to reproduce the issue with cmake only, so I believe it's not related to what I'm trying to do, but rather an issue in how this project builds itself.

System info:

• Ryzen 5900X CPU
• 7900XTX GPU (gfx1100)
• ROCm 6.0.2 from Arch

The reproduction process is as follows:

1. Fresh clone of the repo
2. mkdir build && cd build
3. cmake -GNinja -DWHISPER_HIPBLAS=on -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ ..
4. ninja

And the output that produces on my machine:

[4/20] Building CXX object CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o
FAILED: CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o
/opt/rocm/llvm/bin/clang++ -DGGML_USE_CUDA -DGGML_USE_HIPBLAS -DUSE_PROF_API=1 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -D__HIP_PLATFORM_AMD__=1 -isystem /opt/rocm/include -Wall                                       -Wextra                                     -Wpedantic                                  -Wcast-qual                              -O3 -DNDEBUG -std=gnu++11 -fPIC -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -x hip -MD -MT CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o -MF CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o.d -o CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o -c /tmp/whisper.cpp/ggml-cuda.cu
In file included from /tmp/whisper.cpp/ggml-cuda.cu:5:
/tmp/whisper.cpp/ggml-cuda/common.cuh:3:10: fatal error: 'ggml.h' file not found
#include "ggml.h"
         ^~~~~~~~
1 error generated when compiling for gfx906.
[12/20] Building CXX object CMakeFiles/whisper.dir/whisper.cpp.o
/tmp/whisper.cpp/whisper.cpp:206:29: warning: unused function 'ggml_mul_mat_pad' [-Wunused-function]
static struct ggml_tensor * ggml_mul_mat_pad(struct ggml_context * ctx, struct ggml_tensor * x, struct ggml_tensor * y, int pad = 32) {
                            ^
1 warning generated.
[13/20] Building CXX object examples/server/CMakeFiles/server.dir/server.cpp.o
ninja: build stopped: subcommand failed.

Setting DAMDGPU_TARGETS=gfx1100 produces the same error, so I doubt it's related.

Edit: I'll also add that building with make WHISPER_HIPBLAS=1 doesn't cause any errors.

[lyssieth]

Manually editing the generated build.ninja file's ggml.cuda.cu.o chunk to the following:

build CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o: CXX_COMPILER__ggml-rocm_unscanned_Release /home/lys/Downloads/whisper.cpp/ggml-cuda.cu || cmake_object_order_depends_target_ggml-rocm
  DEFINES = -DGGML_USE_CUDA -DGGML_USE_HIPBLAS -DUSE_PROF_API=1 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -D__HIP_PLATFORM_AMD__=1
  DEP_FILE = CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o.d
  FLAGS = -Wall                                       -Wextra                                     -Wpedantic                                  -Wcast-qual                              -O3 -DNDEBUG -std=gnu++11 -fPIC -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -x hip
  INCLUDES = -isystem /opt/rocm/include -I/home/lys/Downloads/whisper.cpp/. # <--- Edit is here, adding the include for `whisper.cpp/.`
  OBJECT_DIR = CMakeFiles/ggml-rocm.dir
  OBJECT_FILE_DIR = CMakeFiles/ggml-rocm.dir

Makes it build again, but then I get:

./bin/main: symbol lookup error: /home/lys/Downloads/whisper.cpp/build/libwhisper.so.1: undefined symbol: _Z19ggml_cuda_op_im2colR25ggml_backend_cuda_contextP11ggml_tensor

[lyssieth]

The first bug (where ggml.h is not found) can be easily replicated in the rocm/rocm-terminal docker image.

After installing the dependencies (sudo apt update && sudo apt install -y hipblas rocblas)

1. mkdir build && cd build
2. cmake -DWHISPER_HIPBLAS=ON -DWHISPER_BUILD_TESTS=OFF -DWHISPER_BUILD_EXAMPLES=OFF -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER='/opt/rocm/llvm/bin/clang++' ..
3. cmake --build .

And it fails with the ggml.h not found error as expected.