ggml-vicuna-13b-1.1-q4_2.bin

[chongy076]

Expected Behavior

Load and able to run

Current Behavior

llama_model_load: loading model from '...models/llama_cpp/ggml-vicuna-13b-1.1-q4_2.bin' - please wait ... llama_model_load: n_vocab = 32000 llama_model_load: n_ctx = 512 llama_model_load: n_embd = 5120 llama_model_load: n_mult = 256 llama_model_load: n_head = 40 llama_model_load: n_layer = 40 llama_model_load: n_rot = 128 llama_model_load: f16 = 5 llama_model_load: n_ff = 13824 llama_model_load: n_parts = 2 llama_model_load: type = 2 llama_model_load: invalid model file '...models/llama_cpp/ggml-vicuna-13b-1.1-q4_2.bin' (bad f16 value 5) llama_init_from_file: failed to load model llama_generate: seed = 1682654006

Steps to Reproduce

Please provide detailed steps to reproduce the issue.

1. Step 1 -download from https://gpt4all.io/models/ggml-vicuna-13b-1.1-q4_2.bin
2. Step 2 -copy to ...GPT4All\models\llama_cpp
3. Step 3 -run webui and browse http://localhost:9600/ and goto setting and select ggml-vicuna-13b-1.1-q4_2.bin

Possible Solution

If you have any suggestions on how to fix the issue, please describe them here.

• i get to know the model is trained for 3.5 , it need to migrate or convert.

Context

Please provide any additional context about the issue.

Screenshots

If applicable, add screenshots to help explain the issue.

[ParisNeo]

ok, I'll look it up this evening.

[henrik-m]

Same error occurs with the vicuna-7b model as well.

[noctetemp]

same here too

[ParisNeo]

I'm closing this as the new ui solves this problem by using the new llamacpp official backend which support all those new formats.