sapientinc
Support for AMD ROCm devices additionally to NVIDIA CUDA one.
Hi team, I would be thrilled to test and bench this promising new model on various tasks and scenarios that I currently delegate to a traditional transformers based model, but my GPU farm exclusively run on AMD currently.
As you use pytorch, I could try to run it as is, but the requirement for FlashAttention repository is kinda cumbersome in that situation as pytorch does indeed provide flash attention functions and features since 2.2, it may be worthy to swap this requirement with the pytorch available module?
Feel free to let me know if you need further information or anything on that topic!
have you seen / tested https://gist.github.com/robbiemu/50c14bad20eb42dbbbcd6fca0c44889b ?
Not yet nope, didn't knew about it ^^ Is that something noted somewhere on README ?? Asking to find if I just misreaded it ^^
What I’m really wondering is: when does a RISC-V version release? but Native AMD Support is Interesting too
