fast-llama Failed to load model

Failed to load model

Open akhil3417 opened this issue 1 year ago • 1 comments

./main -f gguf -c ../text-generation-webui/models/beagle14-7b.Q5_K_M.gguf

ERROR: [./src/model_loaders/gguf_loader.cpp:263] [load_gguf()] Unsupported file type:17

Failed to load model

Jan 16 '24 12:01 akhil3417

i think u need to use Q8 because it does not support Q5 yet

but i have problem with the tokenizer setting for mistral 7b. not sure how to do it too.

Jan 24 '24 00:01 sprappcom