whisper.cpp icon indicating copy to clipboard operation
whisper.cpp copied to clipboard

Put OpenVINO and OpenBLAS together gives better performance

Open Just-Explode opened this issue 6 months ago • 0 comments

This is just my discovery. idk where should i put it tho For my pc (X1504VA), OpenVINO and OpenBLAS gives significant performance improvement. However following the README.md, it seems that these 2 won't combine together (one belongs to make, the other belongs to cmake), so I copied some stuff from Makefile and put it into src/Cmakelist.txt by the help of ChatGPT. We think up of this piece of code:

if (WHISPER_OPENBLAS)
    set(GGML_OPENBLAS 1)
    set(DEPRECATE_WARNING 1)
    add_definitions(-DGGML_USE_BLAS)

    # Use pkg-config to find OpenBLAS
    find_package(PkgConfig REQUIRED)
    pkg_check_modules(OPENBLAS REQUIRED openblas)

    include_directories(${OPENBLAS_INCLUDE_DIRS})
    link_directories(${OPENBLAS_LIBRARY_DIRS})
    add_compile_options(${OPENBLAS_CFLAGS_OTHER})

    target_link_libraries(whisper PRIVATE ${OPENBLAS_LIBRARIES})

    # Add ggml-blas source file
    target_sources(whisper PRIVATE ${CMAKE_SOURCE_DIR}/ggml/src/ggml-blas.cpp)
endif()

then, we compiled it with cmake -B build -DWHISPER_OPENVINO=1 -DWHISPER_OPENBLAS=1 and cmake --build build -j --config Release

for some reason the main execution won't show that it's capable with BLAS, but in my setup it does. Here's how I make the conclusion. (for aesthetic reason I'll only put the summary here)

here's my main test code: ./main -m /mnt/test/ggml-large-v3.bin -f /mnt/test/whisper.cpp/samples/jfk.wav -oved GPU. Depends on occasion I might change oved to something else. Compile Option, CPU/GPU, Time Spent (unit=sec) OpenVino , GPU , 32 31 OpenVino , CPU , 120 80 50 52 47 45 OpenVino+OpenBLAS, GPU, 26 22 //modified one OpenVino+OpenBLAS, CPU, 40 37 47 //modified one OpenBLAS , CPU , 41 43 no option (pure), CPU , 54,59

So I think BLAS and OpenVINO both worked well. For now I'm happy with this configuration, I think I'll keep it for a while. And hopefully it can help more people (If this is not something severely dangerous(?

Just-Explode avatar Aug 18 '24 09:08 Just-Explode