ggerganov
New Update CPP 1.7.2 and 1.7.3 Is Too Slow in Subtitle Edit
Hello,
Whisper CPP is used in Subtitle Edit to transcribe audio to text. It has been working pretty smoothly until recent updating to CPP 1.7.2 and 1.7.3. The transcribing process goes much slower in both latest CPP updates; for example, it takes more than 26 minutes to complete the process in CPP 1.7.2 and 1.7.3 on a 5 minutes mp3 file with a medium.en Whisper model, much longer than the time of 5~6 minutes in previous CPP versions. A new file 'openblas.dll' is used in CPP 1.7.2 and 1.7.3, but a file named ' liboenblas.dll' is used instead in previous versions. When the file 'openblas.dll' was replaced with the 'liboenblas.dll', renamed as 'openblas.dll', in CPP 1.7.3, the process finished surprisingly within 2.6 minutes. More details discussion about this issue can be found in https://github.com/SubtitleEdit/subtitleedit/issues/9113.
A Whisper log file used in Subtitle Edit and libopenblas.dll are also attached here, hopefully it helps.
cpp.zip
Thank you!
I've also noticed Whisper being much slower in Subtitle Edit recently.
These are some results that I've obtained with the small model on an mp3 audio that is 2 minutes 30 seconds long:
- Subtitle Edit 4.0.10 using Whisper CPP 1.7.3: 5m 15s.
- Subtitle Edit 4.0.9 using Whisper CPP 1.7.2: 7m 55s.
- Subtitle Edit 4.0.8 using Whisper CPP 1.7.1(?): 1m 00s.
I have the same problem. CPP in the latest Subtitle Edit has become very slow, more than 5 times slower than in Subtitle Edit 4.0.8. It was the only fast Whisper engine on devices that don't have RTX. Very sad that it no longer works.
@abc16361 are you using the libopenblas file from version SE 4.0.8 for the SE 4.10.0? Did I understand you correctly?
@abc16361 are you using the
libopenblasfile from version SE 4.0.8 for the SE 4.10.0? Did I understand you correctly?
I am still using SE 4.0.8 with ' libopenblas ', but I just tested SE 4.0.10 with ' libopenblas ' from my SE 4.0.8 and the result is pretty good, less than 4 minutes on a 5 minutes mp3, using medium.en model. The way I did is:
- setup FFmpeg first on a brand new SE 4.0.10;
- download CPP on SE 4.0.10, then 13 files would be downloaded into Cpp folder;
- copy the file ' libopenblas.dll ' into the Cpp folder in SE 4.0.10;
- rename file ' openblas.dll ' as ' openblas.dll bk ' in Cpp folder so that it won't be used;
- then rename ' libopenblas.dll ' as ' openblas.dll ' so that it can be used in SE 4.0.10
- copy medium.en model to Models folder;
- then start to run
The transcript is accurate as that on SE 4.0.8.
@abc16361 thank you very much for the detailed explanation. But I will also stay on version 4.0.8. I'll wait until this problem is fixed.
Yes, but it's difficult for me to make a fix because I don't have a Windows environment. Can you investigate the problem and propose a fix in the form of a PR?
@ggerganov The issue has already been investigated in this thread and is related to the use of openblas.dll instead of libopenblas.dll.
@abc16361 I've taken a look into this and have a possible solution/suggestion. Full details/write-up can be found here
What I believe is happening is that with the change to using OpenBLAS from the vcpgk package manager, we no longer get libopenblas.dll which contains multiple optimized kernels for different CPU arcitectures. This might not be a problem for CI but can effect distributions like yours.
My understanding is that at runtime OpenBLAS will detect the CPU and use the most optimized kernel for it. But openblas.dll is often used by package managers like vcpkg. These may be stripped down versions which don't contains all the kernels (and might lack the detection capabilities as well, this is still not clear to me). If we look at libopenblas.dll it is about 48MB, and openblas.dll is 1.7MB.
I think this issue could be solved by building similar to something like this. Download and unpack OpenBLAS-0.3.29:
Invoke-WebRequest "https://github.com/OpenMathLib/OpenBLAS/releases/download/v0.3.29/OpenBLAS-0.3.29_x64_64.zip" -OutFile "OpenBLAS-0.3.29.zip"
Expand-Archive "OpenBLAS-0.3.29.zip" -DestinationPath "OpenBLAS-0.3.29"
And then build using:
set OPENBLAS_PATH=%cd%\OpenBLAS-0.3.29
cmake -S . -B build -A x64 ^
-DCMAKE_BUILD_TYPE=Release ^
-DGGML_BLAS=ON ^
-DGGML_BLAS_VENDOR=OpenBLAS ^
-DBLAS_LIBRARIES="%OPENBLAS_PATH%/lib/libopenblas.lib" ^
-DBLAS_INCLUDE_DIRS="%OPENBLAS_PATH%/include" ^
-DWHISPER_SDL2=ON ^
-DSDL2_DIR="C:\Users\danie\work\audio\SDL2-devel-2.32.4-VC\SDL2-2.32.4\cmake"
cmake --build build --config Release
copy "%OPENBLAS_PATH%\bin\libopenblas.dll" ".\build\bin\Release"
The linked document contains some more details and how you can build and test this locally. Could you give this a try locally and see if this brings the performance back to the level of whisper.cpp 1.7.1?
I could not figure out how the artifacts that get published to https://github.com/SubtitleEdit/support-files are built, but in this case the system that builds them would need to be updated.
As of writing this comment, I can confirm on Windows you need to manually download prebuilt binaries and include just like shown by @danbev , because vcpkg version of OpenBLAS is indeed not a full version, but a generic build.
Also, installing with dynamic-arch from vcpkg like this doesn't work either: vcpkg install openblas[dynamic-arch]:x64-windows-release as the command fails due to a build failure.
Not only that, when building a Node.js addon and importing it, I need to manually copy libopenblas.dll in the same folder the addon files reside in.
Hi,
On Subtitle Edit 4.0.13 running Whisper CPP 1.7.6, the speed seems to go back to normal. In my test sample (2 minutes 30 seconds long), it's even faster than it was previous to the issue:
Subtitle Edit 4.0.13 running Whisper CPP 1.7.6: 0m 48s. Subtitle Edit 4.0.10 running Whisper CPP 1.7.3: 5m 15s. Subtitle Edit 4.0.9 running Whisper CPP 1.7.2: 7m 55s. Subtitle Edit 4.0.8 running Whisper CPP 1.7.1(?): 1m 00s.
@vivadavid, the issue has indeed been fixed. As you can see, CPP again uses libopenblas.dll, as it did in Subtitle Edit 4.0.8.
@vivadavid, the issue has indeed been fixed. As you can see, CPP again uses
libopenblas.dll, as it did in Subtitle Edit 4.0.8.
That's great news!
I'm happy to hear that this is working again, and sorry about the delay in providing a fix. Closing this issue now.