Unsupported Model : Zephyr 'stablelm' GGUF

Open Jonathanjordan21 opened this issue 2 years ago • 0 comments

I wonder why Zephyr is not yet supported in ctransformers.

from ctransformers import AutoModelForCausalLM

m = AutoModelForCausalLM.from_pretrained('TheBloke/stablelm-zephyr-3b-GGUF', model_file='stablelm-zephyr-3b.Q4_K_M.gguf')

The above code actually downloaded the file from huggingface repo, but it output this error

RuntimeError: Failed to create LLM 'stablelm' from '/root/.cache/huggingface/hub/models--TheBloke--stablelm-zephyr-3b-GGUF/blobs/74b2613b6e89d904a2ea38d56d233e4d2ca2fe663844b2e7aa90e769d359061b'.

Dec 14 '23 09:12 Jonathanjordan21