AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard

Auto detect if GPU is unsupported

Open rsxdalv opened this issue 11 months ago • 1 comments

The error:

RuntimeError: CUDA error: no kernel image is available for execution on the device

Implies but does not immediately explain that it's due to old architecture. I propose either displaying a warning such as "The GPU might fail" if Compute Capability is below 7.5 or running any torch command that will fail and if it fails displaying "The GPU will fail" sort of message.

rsxdalv avatar Mar 15 '24 22:03 rsxdalv

Ran into the same problem and figured out it's because the GPU card isn't supported. For people who wish to try AWQ quantized models, you can check your GPU's compute capability on this CUDA website: NVIDIA CUDA GPUs. Might save some time for anyone else bumping into this issue.

LxYuan0420 avatar Mar 28 '24 06:03 LxYuan0420