whisper.cpp
whisper.cpp copied to clipboard
Put OpenVINO and OpenBLAS together gives better performance
This is just my discovery. idk where should i put it tho For my pc (X1504VA), OpenVINO and OpenBLAS gives significant performance improvement. However following the README.md, it seems that these 2 won't combine together (one belongs to make, the other belongs to cmake), so I copied some stuff from Makefile and put it into src/Cmakelist.txt by the help of ChatGPT. We think up of this piece of code:
if (WHISPER_OPENBLAS)
set(GGML_OPENBLAS 1)
set(DEPRECATE_WARNING 1)
add_definitions(-DGGML_USE_BLAS)
# Use pkg-config to find OpenBLAS
find_package(PkgConfig REQUIRED)
pkg_check_modules(OPENBLAS REQUIRED openblas)
include_directories(${OPENBLAS_INCLUDE_DIRS})
link_directories(${OPENBLAS_LIBRARY_DIRS})
add_compile_options(${OPENBLAS_CFLAGS_OTHER})
target_link_libraries(whisper PRIVATE ${OPENBLAS_LIBRARIES})
# Add ggml-blas source file
target_sources(whisper PRIVATE ${CMAKE_SOURCE_DIR}/ggml/src/ggml-blas.cpp)
endif()
then, we compiled it with cmake -B build -DWHISPER_OPENVINO=1 -DWHISPER_OPENBLAS=1
and cmake --build build -j --config Release
for some reason the main execution won't show that it's capable with BLAS, but in my setup it does. Here's how I make the conclusion. (for aesthetic reason I'll only put the summary here)
here's my main test code: ./main -m /mnt/test/ggml-large-v3.bin -f /mnt/test/whisper.cpp/samples/jfk.wav -oved GPU
. Depends on occasion I might change oved to something else.
Compile Option, CPU/GPU, Time Spent (unit=sec)
OpenVino , GPU , 32 31
OpenVino , CPU , 120 80 50 52 47 45
OpenVino+OpenBLAS, GPU, 26 22 //modified one
OpenVino+OpenBLAS, CPU, 40 37 47 //modified one
OpenBLAS , CPU , 41 43
no option (pure), CPU , 54,59
So I think BLAS and OpenVINO both worked well. For now I'm happy with this configuration, I think I'll keep it for a while. And hopefully it can help more people (If this is not something severely dangerous(?