Eric Hartford
Eric Hartford
here is full stack trace: ``` Traceback (most recent call last): File "/home/eric/git/qlora/qlora.py", line 845, in train() File "/home/eric/git/qlora/qlora.py", line 807, in train train_result = trainer.train() File "/home/eric/miniconda3/envs/qlora/lib/python3.10/site-packages/transformers/trainer.py", line 1539,...
I got around that problem by moving the patch code to an earlier point before loading the model. But I hit another error: ```RuntimeError: FlashAttention only support fp16 and bf16...
here is full stack trace: ``` Traceback (most recent call last): File "/home/eric/git/qlora/qlora.py", line 845, in train() File "/home/eric/git/qlora/qlora.py", line 807, in train train_result = trainer.train() File "/home/eric/miniconda3/envs/qlora/lib/python3.10/site-packages/transformers/trainer.py", line 1539,...
could be possible that qlora patches it again after I patch it?
ok trying
RuntimeError: FlashAttention only support fp16 and bf16 data type
seems maybe FlashAttention needs to be modified to support this
@tridao is it possible to merge this to support ROCm? https://github.com/ROCmSoftwarePlatform/flash-attention
I doubt they would disapprove of merging, Seems just a rift of communication. I will reach out.
I have mi100s, would love to be able to use them