[Feature Request] Add AMD ROCm Support

[cotsuka]

llama.cpp supports ROCm, which would open this project to be used on AMD hardware as well: https://github.com/ggerganov/llama.cpp#hipblas

[agates]

Happy to test on a RX 6800 XT if needed.

[hydragyrum32]

Also willing to test, have a 6600 and a 6400 here

[tagyro]

+1 for testing this, I have a 6900XT

[AnttiRae]

Willing to help testing with my 7900XTX

[jykae]

Possibly quite simple to implement as there are good examples for CUDA https://github.com/getumbrel/llama-gpt/tree/master/cuda

[ParthJadhav]

Hey @jykae @AnttiRae @tagyro @hydragyrum32 @agates @cotsuka .

https://github.com/getumbrel/llama-gpt/pull/114

I need help testing this, I don't have an AMD GPU but I've created a POC for AMD Support. Please test if possible and let me know in the comments so I can make the required changes.

[jykae]

@AnttiRae check out, looks like there's candidate for ROCm support now 😊