flash-attention icon indicating copy to clipboard operation
flash-attention copied to clipboard

Published 20 hours ago • verified publisher icon Dao-AILab

There is no cu123 but cu124 for PyTorch

Open nasyxx opened this issue 1 year ago • 6 comments

Is there a mistake for the released whl?

Screenshot 2024-07-24 at 9 57 06 PM

nasyxx avatar Jul 25 '24 02:07 nasyxx

No mistake

tridao avatar Jul 25 '24 03:07 tridao

No mistake

Can we get a cuda 124 version. As pytorch supports 12.4 but 12.3.

nasyxx avatar Jul 27 '24 05:07 nasyxx

12.3 will work

tridao avatar Jul 27 '24 06:07 tridao

It doesnt. Neither would 12.1 https://github.com/Dao-AILab/flash-attention/issues/1093

coldn00dles avatar Jul 27 '24 22:07 coldn00dles

cuda minor version are compatible

tridao avatar Jul 27 '24 22:07 tridao

II've compiled a version that works with CUDA 12.4 and PyTorch 2.4 for Windows.  You can find the wheel at this address: https://huggingface.co/windzmf/flash_attn-2_6_3_cu124torch2_4cxx11_win_amd64.whl/tree/main  This wheel is specifically compiled for:

  • Flash Attention version 2.6.3
  • CUDA 12.4
  • PyTorch 2.4
  • Windows AMD64 platform
  • Python 3.11 ============================================================= 我编译了一个适用于CUDA 12.4和PyTorch 2.4的Windows版本。 您可以在以下地址找到： https://huggingface.co/windzmf/flash_attn-2_6_3_cu124torch2_4cxx11_win_amd64.whl/tree/main -Flash attention 2.6.3 -CUDA 12.4 -PyTorch 2.4 -Windows AMD64平台 -Python 3.11

AdamWhiteC avatar Sep 14 '24 16:09 AdamWhiteC