whisper.cpp
whisper.cpp copied to clipboard
ggerganov
[Question] Does Cuda Toolkit need to be installed by end-user when using WHISPER_CUBLAS=1?
Hi I'm pretty new to AI stuff. I built the whisper.cpp with cublas flag enabled to generate a dynamic library (.dll) that my application can use. My plan is to develop a software for windows (and later doing separate build for macOS without cublas flag)
While I have Cuda Toolkit installed on my local computer, I'm wondering whether these cuBLAS libraries are statically linked to the whisper.cpp dll or not. So my questions:
- Does my end-user need to install the Cuda Toolkit to use my software?
- If they need to install it (or include cuda dll files) in order to use Cuda, will the library fallback to use the CPU without Cuda Toolkit installed? I would prefer my software can continue to run without cuda toolkit or do a separate build
- EDIT: Also, I'm using cuBLAS 12.3, If user required to install the toolkit, does this mean they have to install the matching version as the one that was used for the build? If so, can I distribute the required DLL files without the user installing the whole toolkit?
Does my end-user need to install the Cuda Toolkit to use my software?
No, once the program is compiled and bundled with the runtime libraries, your end users only need to install NVIDIA drivers.
will the library fallback to use the CPU without Cuda Toolkit installed?
GGML will automatically switch to using the CPU if GPU drivers are not installed.
If user required to install the toolkit, does this mean they have to install the matching version as the one that was used for the build?
They need to install a driver that is compatible with your CUDA toolkit.
Thanks for your answer!
once the program is compiled and bundled with the runtime libraries
Sorry, just wanted to be absolutely certain here, the "program" you're referring here is the whisper.cpp library or my program? and what do you mean by bundled?
I plan to create a portable and standalone program, so I would prefer to ship as little files as possible, I'm thinking to ship the executable and the whisper.dll without anything else. Assuming I'm using the cuBLAS flag enabled when building whisper.dll, would that be enough to use the cuBLAS?