flashinfer icon indicating copy to clipboard operation
flashinfer copied to clipboard

Published 20 hours ago • verified publisher icon flashinfer-ai

CUDA Error: no kernel image is available for execution on the device (209) /tmp/build-via-sdist-nl8se4dx/flashinfer-0.0.4+cu118torch2.2/include/flashinfer/attention/decode.cuh: line 871 at function cudaFuncSetAttribute(kernel, cudaFuncAttributeMaxDynamicSharedMemorySize, smem_size)

Open lucasjinreal opened this issue 9 months ago • 2 comments

Does the cudnn can not be v9?

lucasjinreal avatar May 16 '24 02:05 lucasjinreal

This is due to CUDA version mismatch, what's the CUDA version (and PyTorch CUDA version) on your device?

nvidia-smi
python -c "import torch; print(torch.version.cuda)"

yzh119 avatar May 16 '24 04:05 yzh119

Am using cuda118 ,

>>> import torch
torch>>> torch.__version__
'2.2.0+cu118'

installed with pip install flashinfer -i https://flashinfer.ai/whl/cu118/torch2.2/ my torch works ok (even though torch now using their own nvidia toolkit through pypi)

Doe sthey any miniml func can be test why the cuda is mismatch?

lucasjinreal avatar May 16 '24 06:05 lucasjinreal