buzz
buzz copied to clipboard
chidiwilliams
Whisper transcription super slow - Mac Silicon
Running on an M1 Mac - tested on Whisper, Model small. Transcription, English selected. I assume it was using the small-en model but not sure.
File successfully outputted but after 20:52. File is a 30 minute mp3 file.
Ran the exact same file but with a python script I wrote which calls the whisper small-en model - it took 04:32. Now the only difference between the two scenarios that I can tell is that Buzz outputted one with line by line timestamps (word level timings were off in both cases). But why is it so much slower?
I will test whisper.cpp once the new changes about that are compiled into a binary (running Buzz v. 0.7.0 which is what brew installed) - it looks like there have been commits (e.g. #331 but no new release yet.)
Indeed. Processing an 1-hour-long video using large model took me hours... Buzz seems only use 1 CPU core on my M1 Max
Indeed. Processing an 1-hour-long video using large model took me hours... Buzz seems only use 1 CPU core on my M1 Max
Yep I noticed the same :-( Was so excited to check the latest version I installed it and ran the transcription remotely while being in a hotel room 3000km away from home :-). My hope is that now that the swift version is complete it can be tweaked and optimized to access more fully M1 and M2 architectures which wouldn't happen without swift.
The new version is now live on the App Store (for $9.99). It uses Whisper.cpp optimized for Mac silicon and also has search, audio playback, supports the Whisper API, and many other new features.
The new version is now live on the App Store (for $9.99). It uses Whisper.cpp optimized for Mac silicon and also has search, audio playback, supports the Whisper API, and many other new features.
That's great but I'm not sure why I would pay for it since it wasn't working just a short time ago and has been pretty buggy. Perhaps make a free version available for beta testers?
Sure, you can join the test flight here: https://github.com/chidiwilliams/buzz/discussions/374