Error: "is this really a GGML file?"

[bobhairgrove]

Using latest model file "ggml-model-q4_0.bin" from llama.cpp, I get these errors (source_documents contains the SOTU sample file):

~/code/privateGPT/privateGPT-main$ python3 ingest.py
Loading documents from source_documents
Loaded 1 documents from source_documents
Split into 90 chunks of text (max. 500 tokens each)
llama.cpp: loading model from /home/bob/code/privateGPT/models/ggml-model-q4_0.bin
error loading model: unknown (magic, version) combination: 67676a74, 00000002; is this really a GGML file?
llama_init_from_file: failed to load model
Traceback (most recent call last):
  File "/home/bob/.local/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 78, in validate_environment
    values["client"] = Llama(
  File "/home/bob/.local/lib/python3.10/site-packages/llama_cpp/llama.py", line 159, in __init__
    assert self.ctx is not None
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bob/code/privateGPT/privateGPT-main/ingest.py", line 62, in <module>
    main()
  File "/home/bob/code/privateGPT/privateGPT-main/ingest.py", line 53, in main
    llama = LlamaCppEmbeddings(model_path=llama_embeddings_model, n_ctx=model_n_ctx)
  File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1102, in pydantic.main.validate_model
  File "/home/bob/.local/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 98, in validate_environment
    raise NameError(f"Could not load Llama model from path: {model_path}")
NameError: Could not load Llama model from path: /home/bob/code/privateGPT/models/ggml-model-q4_0.bin

Running this directly in llama.cpp seems to work OK:

# run the inference
./main -m ./models/7B/ggml-model-q4_0.bin -n 128

Here is my .env file:

PERSIST_DIRECTORY=/home/bob/code/privateGPT/db
LLAMA_EMBEDDINGS_MODEL=/home/bob/code/privateGPT/models/ggml-model-q4_0.bin
MODEL_TYPE=LlamaCpp
MODEL_PATH=/home/bob/code/privateGPT/models/koala-7B.ggml.q4_0.bin
MODEL_N_CTX=1000

These files DO EXIST in their directories as quoted above.

Anybody know what is the issue here?

[proming]

upate llama-cpp-python to 0.1.50 pip3 install -U llama-cpp-python=0.1.50

[shaunxu]

updated to llama-cpp-python to 0.1.57 to get this error resolved.