Jeffser
Add GPU Support for AMD/Intel (Vulkan) – The Only Thing Missing
Hi Jeffser,
First of all — Alpaca is by far the best local LLM app. Clean UI, fast, no fluff. But there's one big missing feature: GPU support for AMD and Intel users.
Right now, Alpaca only uses CPU. While llama.cpp supports CUDA (NVIDIA) and even ROCm (limited AMD), most AMD GPUs like the RX 6600M aren't supported. ROCm is too limited and fragile.
The good news is: llama.cpp now has Vulkan backend, which works with AMD and Intel GPUs cross-platform.
Apps like GPT4All already support GPU via Vulkan out of the box. If Alpaca added Vulkan support, it would be complete — and faster for everyone without NVIDIA cards.
🔧 Suggestion: Ship a build of Alpaca with LLAMA_VULKAN=1 compiled.
Automatically enable Vulkan backend if CUDA isn’t available.
Optionally expose a CLI or config toggle for advanced users.
This one feature would open the door to a much larger user base. Happy to test or help if needed.
Thanks!
Hi thanks for the suggestion but Alpaca doesn't use llama.cpp, instead it uses Ollama which doesn't have vulkan support yet
Shouldn't take long anymore, Ollama largely uses llama.cpp behind the scenes and if they adopt it, this could be doable.
Shouldn't take long anymore, Ollama largely uses llama.cpp behind the scenes and if they adopt it, this could be doable.
As mentioned in the Ollama repository, it does not appear that they will implement it. There was a pull request months ago, but as the author of the pull request mentions, they do not know if they will even implement it, so it has fallen behind with updates.
