Error when compiling with CuBLAS

[ClaraSev1]

I am trying to run whisper.cpp on an Nvidia Jetson Orin Nano module, on which I have CUDA 11.4 installed. When I try to compile by running

make clean
WHISPER_CUBLAS=1 make -j

I get the following error:

clara@NVIDIA-Orin-Nano:~/whisper.cpp$ WHISPER_CUBLAS=1 make -j
I whisper.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  aarch64
I UNAME_M:  aarch64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include
I LDFLAGS:   -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L/targets/aarch64-linux/lib -L/usr/lib/wsl/lib
I CC:       cc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
I CXX:      g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

nvcc --forward-unknown-to-host-compiler -arch=all -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include -Wno-pedantic -c ggml-cuda.cu -o ggml-cuda.o
cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include   -c ggml.c -o ggml.o
cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include   -c ggml-alloc.c -o ggml-alloc.o
cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include   -c ggml-backend.c -o ggml-backend.o
nvcc fatal   : Value 'all' is not defined for option 'gpu-architecture'
cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include   -c ggml-quants.c -o ggml-quants.o
make: *** [Makefile:233: ggml-cuda.o] Error 1
make: *** Waiting for unfinished jobs....

When I try to compile it again, but this time without running make clean first, I get

clara@NVIDIA-Orin-Nano:~/whisper.cpp$ WHISPER_CUBLAS=1 make -j
I whisper.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  aarch64
I UNAME_M:  aarch64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include
I LDFLAGS:   -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L/targets/aarch64-linux/lib -L/usr/lib/wsl/lib
I CC:       cc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
I CXX:      g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

nvcc --forward-unknown-to-host-compiler -arch=all -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include -Wno-pedantic -c ggml-cuda.cu -o ggml-cuda.o
g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/aarch64-linux/include -c whisper.cpp -o whisper.o
nvcc fatal   : Value 'all' is not defined for option 'gpu-architecture'
make: *** [Makefile:233: ggml-cuda.o] Error 1
make: *** Waiting for unfinished jobs....

It might be due to the nvcc command, but I don't know how to fix the error. Does anyone have any ideas?

[MathiasSchindler]

This might be completely unrelated: I see that gcc 9.4 was released almost three years ago. Would you be able to test this with a more recent gcc (9.5.) from the same branch or 10/11/12 that are actively maintained?

[primenko-v]

I had a similar problem when building Whisper on Jetson Nano with CUDA 10.2. It helped to use CMake:

    mkdir build
    cd build
    cmake -DWHISPER_CUBLAS=1  ..
    make -j4

For the latest Whisper version the flag has to be -DWHISPER_CUDA=1, though.