How is the GPU model development progressing?

[martjay]

GPU mode is important for AI

[EL-M3ndy]

ya, please add it

[AlmostHuman34]

Mr.Williams tried but he wasn't able to implement it. With the current state of Buzz Whisper takes way too long for me, 2h audio in 3-4 hours medium model, if it was for gpu I think it would take somewhere around 30-50 minutes with large model.

[martjay]

Mr.Williams tried but he wasn't able to implement it. With the current state of Buzz Whisper takes way too long for me, 2h audio in 3-4 hours medium model, if it was for gpu I think it would take somewhere around 30-50 minutes with large model.

There is now a faster way than the official model.

https://github.com/guillaumekln/faster-whisper https://github.com/ggerganov/whisper.cpp

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[RokeJulianLockhart]

https://github.com/chidiwilliams/buzz/issues/375#issuecomment-1552466795

Die, @stale bot.

[raivisdejus]

Latest development versions of Buzz support GPU for Whisper, Huggingface and Faster Whisper models. I have tested it on Linux. To get latest version on Linux use sudo snap install buzz --channel latest/edge.

Windows and macOS builds of latest development version are available from GitHub Actions. Log into GitHub and download latest development version from some successful GitHub Action https://github.com/chidiwilliams/buzz/actions?query=branch%3Amain

Feedback on GPU support on non-Linux would be welcome.