alpaca-lora
alpaca-lora copied to clipboard
tloen
AMD GPU
Will amd gpu's be supported?
Pretty sure this issue has to get solved first in the bitsandbytes library before this will work on an AMD GPU, but I could be wrong.
Will amd gpu's be supported?
Pretty sure this issue has to get solved first in the bitsandbytes library before this will work on an AMD GPU, but I could be wrong.
I see, I've been having issues trying to get ROcM installed on Ubuntu, been following tutorials all day.
Would you be able to guide me through it and which version to use? (Using ubuntu from the ms store) thanks! (off topic)
ROCM has come a long way and pytorch has good support for it.
Really, the easiest way to use it is to install one of the docker containers: https://hub.docker.com/r/rocm/pytorch
You can follow this video (starting at step 2) and keeping in mind an older version of the docker container was being used: https://youtu.be/d_CgaHyA_n4?t=151
There is also a version of bitsandbytes for rocm that works (i've tried it): https://github.com/broncotc/bitsandbytes-rocm
If you can get that installed and running, you might be able to train a model on an AMD card without having to wait for official AMD support in the bitsandbytes library.
ROCM has come a long way and pytorch has good support for it.
Really, the easiest way to use it is to install one of the docker containers: https://hub.docker.com/r/rocm/pytorch
You can follow this video (starting at step 2) and keeping in mind an older version of the docker container was being used: https://youtu.be/d_CgaHyA_n4?t=151
There is also a version of bitsandbytes for rocm that works (i've tried it): https://github.com/broncotc/bitsandbytes-rocm
If you can get that installed and running, you might be able to train a model on an AMD card without having to wait for official AMD support in the bitsandbytes library.
Which docker image would I use for a radeon rx 6700xt?
Which docker image would I use for a radeon rx 6700xt?
Grab the latest pytorch one here: https://hub.docker.com/r/rocm/pytorch
I had good success with: rocm5.4.1_ubuntu20.04_py3.7_pytorch_1.12.1
Also, it looks like the BitsAndBytes lib has been patched to fully work on rocm here: https://github.com/Titaniumtown/bitsandbytes-rocm/tree/patch-2
So use the patched repo to install bitsandbytes for rocm.
Let me know if you get this working on your AMD card..
One more tip, you since you are on a 6700xt, you'll probably have to set this ENV variable to fake your GPU:
HSA_OVERRIDE_GFX_VERSION=10.3.0
https://github.com/TimDettmers/bitsandbytes/issues/107#issuecomment-1824892416
