Import Error

[Techinix]

Used to work fine : Python 3.10.12 , torch 2.2.1+cu121 undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

[saurabh-kataria]

I am also getting ImportError: <path-to-conda-env>/lib/python3.8/site-packages/flash_attn_2_cuda.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

[nzw0301]

I could resolve the error by installing flash attention from source on A30 machine with PyTorch 2.3.

[Galaxy-Husky]

I am also getting ImportError: <path-to-conda-env>/lib/python3.8/site-packages/flash_attn_2_cuda.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

I got the same error with PyTorch 2.3.0 when I installed it with the wheel file. I solved the problem by compiling from the source, which means git clone the repo and run python setup.py install.

[andreaskoepf]

The _ZN3c104cuda9SetDeviceEi import error seems to be fixed with v2.5.8, i.e. latest pip package of flash-attn==2.5.8 works with PyTorch 2.3.0! thnx a lot @tridao !

[comaniac]

Hey I got the same error with PyTorch 2.3.0 and flash-attn==2.5.8. Is there anything changed in the wheel?

flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

[zhangyu68]

I am also getting ImportError: <path-to-conda-env>/lib/python3.8/site-packages/flash_attn_2_cuda.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

I got the same error with PyTorch 2.3.0 when I installed it with the wheel file. I solved the problem by compiling from the source, which means git clone the repo and run python setup.py install.

helpful,thank you