linuxserver
[FEAT] AMD GPU ROCm Support
Is this a new feature request?
- [x] I have searched the existing issues
Wanted change
I would like another image with AMD ROCm support added. To make it faster on amd machines.
Thank you!
Reason for change
to make whisper faster on amd machines.
Proposed code change
No response
Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.
rocm is like cuda it is massive and it needs to be inside the container there is no runtime to pass the host install to the container, its like 4 gigs most people just base on their baseimages like rocm/rocm-terminal.
So basically no, we are not planning on adding 4 gigs to either the baseimages or this container, it sucks that this is how it is implemented and it walls stuff out, but maybe a docker mod that runs on init ? Just spitballing a possible solution.
Could you not make a :rocm container one, and base it on your base? So then its not making everyone's images that much larger? Then us with amd can get the rocm image.
Sure it is possible but I am just chiming in as the why as a team member. I do not maintain this image and I do not use this program, just thought I would give you information.
You also realize that doing this without a cuda one is pointeless as nvidia has something like 90% of the GPU share?
You starting to see how this becomes a big cumbersome thing to support ?
This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.
I'll consider it, but I need to do a bunch of testing because I have no idea what's actually required to support rocm with faster-whisper and I've only got a RX 7700S to play with, which doesn't technically have rocm support (even though it seems to mostly work).
localai added gpu support for faster-whisper:
Re.: https://github.com/mudler/LocalAI?tab=readme-ov-file#hardware-acceleration-matrix
They have a model ready to download and the base also ready.
Repo with igpu intel: https://github.com/tannisroot/wyoming-whisper-cpp-intel-gpu-docker
I also find this: https://github.com/roryeckel/wyoming_openai
I will test and switch for one that I was able to use openai API and wyoming to use only just one endpoint. Maybe consider one of these to switch in future.
