AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard
verified publisher icon casper-hansen

RuntimeError: CUDA error: no kernel image is available for execution on the device

Open noaebbot opened this issue 1 year ago • 2 comments

I get this error: RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

This is my driver: Skärmavbild 2024-08-13 kl 12 02 38

cuda-toolkit 12.1.0 0 nvidia torch 2.4.0 pypi_0 pypi

noaebbot avatar Aug 13 '24 10:08 noaebbot

The error comes when i run model.generate. I have tried a variety of different cuda versions (12.1, 12.4, 12.6), but no one seems to work. Is it just that my GPU (tesla t4) is not compatible with the awq library?

noaebbot avatar Aug 14 '24 15:08 noaebbot

same error in Tesla V100, but not appears in NVIDIA A100. it seems the older device has something weird.

huangfu170 avatar Mar 16 '25 11:03 huangfu170