CUDA error 222 at D:\a\ctransformers\ctransformers\models\ggml\ggml-cuda.cu:6045: the provided PTX was compiled with an unsupported toolchain.

[AnhNgDo]

Hello, I'm trying to use ctransformers as below:

from ctransformers import AutoModelForCausalLM

# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
llm = AutoModelForCausalLM.from_pretrained("TheBloke/zephyr-7B-alpha-GGUF", model_file="zephyr-7b-alpha.Q4_K_M.gguf", model_type="mistral", gpu_layers=50)

print(llm("AI is going to"))

Got this error: CUDA error 222 at D:\a\ctransformers\ctransformers\models\ggml\ggml-cuda.cu:6045: the provided PTX was compiled with an unsupported toolchain.

Not sure where to go from here :( Any help will be much appreciated!

[AlexBlack2202]

i have same problem, and it is not fix

as i guest that GGUF format doesn't support gpu

[parth-verma7]

i have same problem, and it is not fix

as i guest that GGUF format doesn't support gpu

Did u find the solution??