NameError: Could not load Llama model from path

[lolxdmainkaisemaanlu]

Traceback (most recent call last): File "c:\Users\Siddhesh\Desktop\llama.cpp\langchain_test.py", line 10, in llm = LlamaCpp(model_path="C:\Users\Siddhesh\Desktop\llama.cpp\models\ggml-model-q4_0.bin") File "pydantic\main.py", line 339, in pydantic.main.BaseModel.init File "pydantic\main.py", line 1102, in pydantic.main.validate_model File "C:\Users\Siddhesh\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\llms\llamacpp.py", line 117, in validate_environment raise NameError(f"Could not load Llama model from path: {model_path}") NameError: Could not load Llama model from path: C:\Users\Siddhesh\Desktop\llama.cpp\models\ggml-model-q4_0.bin

I have tried with raw string, double \, and the linux path format /path/to/model - none of them worked.

The path is right and the model .bin file is in the latest ggml model format. The model format for llamacpp was recently changed from ggml to ggjt and the model files had to be recoverted into this format. Is the issue being caused because of this change?

[MatchTerm]

Same, keep getting the error.

[MauGal]

I too have the same type of error. I tried different models: ggml q4_1 or q5_1.

Info: SO: Windows 11 Python: 3.11 langchain: 0.0.158 llama-cpp-python: 0.1.30

[sharansabi]

did you figure it out?

[MoXiaoluo]

the same question I met.

[masonbrothers]

I suspect that it has to do with running a model with CPU vs GPU. I was able to run a model with the CPU build of llama-cpp-python, but not the CUBLAS build.

[sharansabi]

@masonbrothers how did you make the switch?

[bharrisp9]

pip3 install llama-cpp-python==0.1.49

[Xiang-Pan]

Same error.

[elmarhuseynov]

pip3 install llama-cpp-python==0.1.49

It fixed same error. Thank you

[MatchTerm]

Ok, so just do that pip command and everything's fixed?

[elmarhuseynov]

Yes, but works to slow. I use windows 10 and try to fix.

[yahyaelganyni1]

The command didn't help me I still have the same issue

Iam using StableBeluga 7B - GGML model link

File [~/.local/lib/python3.10/site-packages/pydantic/main.py:342](https://file+.vscode-resource.vscode-cdn.net/home/yahya/ist/test-llama/~/.local/lib/python3.10/site-packages/pydantic/main.py:342), in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for LlamaCppEmbeddings
__root__
  Could not load Llama model from path: [/home/yahya/ist/test-llama/stablebeluga-7b.ggmlv3.q2_K.bin.](https://file+.vscode-resource.vscode-cdn.net/home/yahya/ist/test-llama/stablebeluga-7b.ggmlv3.q2_K.bin.) Received error  (type=value_error)

I have tried everything

[link-king]

me too

[mandrije]

I'm not sure all of these issues are actually the same, but I've hit the error shown above by @yahyaelganyni1 yesterday and noticed the following on llama-cpp-python home page:

I want to try utilizing GPU so I was following these useful instructions to get GPU/CUDA support (along with these) and got latest llama-cpp-python==0.1.83 installed. But after downgrading to the latest version BEFORE the critical/breaking one (i.e. llama-cpp-python==0.1.78), the error above has gone. But I could never conclude that based on the error message... Hope this will help someone.

[mandrije]

Btw, I've just noticed people are referring to the same issue/resolution elsewhere :)

[dosubot[bot]]

Hi, @lolxdmainkaisemaanlu,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue is a NameError related to loading the Llama model from a specified path. There have been discussions about potential causes, including changes in the model format from 'ggml' to 'ggjt', and various attempted solutions such as switching between CPU and GPU builds, and downgrading the llama-cpp-python package to different versions. The current status of the issue is unresolved.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[kanamadiharshita]

cmake version 3.29 python3.12 llama-cpp-python=0.2.57 llm = Llama( model_path="./PG_dir/mistral-7b-instruct-v0.1.Q4_K_M.gguf", # model_type="mistral", # temperature=0.3, # max_new_tokens = 50 config={"max_new_tokens": 500, "context_length": 4096, "temperature": 0.3},

)

i am getting error ValueError: Model path does not exist: ./PG_dir/mistral-7b-instruct-v0.1.Q4_K_M.gguf