invalid model file?

[Saahil-exe]

D:\AI\PrivateGPT\privateGPT>python privategpt.py Using embedded DuckDB with persistence: data will be stored in: db Found model file at models/ggml-gpt4all-j-v1.3-groovy.bin Invalid model file ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ D:\AI\PrivateGPT\privateGPT\privategpt.py:82 in │ │ │ │ 79 │ │ 80 │ │ 81 if name == "main": │ │ ❱ 82 │ main() │ │ 83 │ │ │ │ D:\AI\PrivateGPT\privateGPT\privategpt.py:38 in main │ │ │ │ 35 │ │ case "LlamaCpp": │ │ 36 │ │ │ llm = LlamaCpp(model_path=model_path, n_ctx=model_n_ctx, n_batch=model_n_bat │ │ 37 │ │ case "GPT4All": │ │ ❱ 38 │ │ │ llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', n_batch=m │ │ 39 │ │ case _default: │ │ 40 │ │ │ print(f"Model {model_type} not supported!") │ │ 41 │ │ │ exit; │ │ │ │ D:\AI\PrivateGPT\privateGPT\pydantic\main.py:341 in pydantic.main.BaseModel.init │ │ │ │ [Errno 2] No such file or directory: 'D:\AI\PrivateGPT\privateGPT\pydantic\main.py' │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ ValidationError: 1 validation error for GPT4All root Unable to instantiate model (type=value_error)

[Elicherla01]

@Saahil-exe, Please share folder strcuture and contends on the .env file

[JasonMaggard]

Looks like this is related to the latest gpt4all. I was able to downgrade to gpt4all==0.2.3 and the error went away.

[FearTheBadger]

I was getting the same error.

Downgrading gpt4all got me past this error, but failed elsewhere (probably my issue):

python3 install --user gpt4all==0.2.3

PERSIST_DIRECTORY=db
MODEL_TYPE=GPT4All
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1000
MODEL_N_BATCH=8
TARGET_SOURCE_CHUNKS=4

$ find . | grep -v .git
1:.
105:./.pre-commit-config.yaml
106:./LICENSE
107:./constants.py
108:./source_documents
109:./source_documents/state_of_the_union.txt
110:./README.md
111:./example.env
112:./ingest.py
113:./requirements.txt
114:./.env
115:./__pycache__
116:./__pycache__/constants.cpython-39.pyc
117:./__pycache__/constants.cpython-311.pyc
118:./db
119:./db/index
120:./db/index/index_b7e0893b-21d9-4718-9c5c-eda5b6b62f8e.bin
121:./db/index/id_to_uuid_b7e0893b-21d9-4718-9c5c-eda5b6b62f8e.pkl
122:./db/index/uuid_to_id_b7e0893b-21d9-4718-9c5c-eda5b6b62f8e.pkl
123:./db/index/index_metadata_b7e0893b-21d9-4718-9c5c-eda5b6b62f8e.pkl
124:./db/chroma-embeddings.parquet
125:./db/chroma-collections.parquet
126:./models
127:./models/ggml-gpt4all-j-v1.3-groovy.bin
128:./privateGPT.py

[andrewdavid48]

@FearTheBadger what's the other issue you get? Asking because I can confirm that downgrading works, but every time I try to query, it fails.

[FearTheBadger]

This issue can probably be closed.

0.3.2 was yanked and reverted back to 0.3.0

https://pypi.org/project/gpt4all/#history

[FearTheBadger]

@FearTheBadger what's the other issue you get? Asking because I can confirm that downgrading works, but every time I try to query, it fails.

I get an Illegal instruction (core dumped) error, when trying to run. But I am doing off the path things like using RHEL9 in a VM. So I am guessing I am the issue.

[andrewdavid48]

I get an Illegal instruction (core dumped) error, when trying to run. But I am doing off the path things like using RHEL9 in a VM. So I am guessing I am the issue.

Got it. Thanks for sharing. Unfortunately I'm not acquainted enough with python or private gpt to help, sorry.

[andrewdavid48]

This issue can probably be closed.

0.3.2 was yanked and reverted back to 0.3.0

https://pypi.org/project/gpt4all/#history

Are you able to run with 0.3.0? Asking because I'm getting the same error, Invalid model file (yes, the model exists, with 0.2.3 I can run properly)

[FearTheBadger]

I am running into my same issue with 0.3.0, but it did get past the posted error.

[wjvander]

I just updated gpt4all==0.3.3 as it now shows here https://pypi.org/project/gpt4all/#history and it seems to be working now, at least with the snoozy model

[andrewdavid48]

I just updated gpt4all==0.3.3 as it now shows here https://pypi.org/project/gpt4all/#history and it seems to be working now, at least with the snoozy model

I'm curious what changes you've made. Using the latest main, still can't get the model to load, even tried using snoozy. gpt4all is installed with 0.3.3

.env file

PERSIST_DIRECTORY=db
MODEL_TYPE=GPT4All
# MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
MODEL_PATH=models/ggml-gpt4all-l13b-snoozy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1024
MODEL_N_BATCH=8
TARGET_SOURCE_CHUNKS=4

Error:

Using embedded DuckDB with persistence: data will be stored in: db
Found model file at  models/ggml-gpt4all-l13b-snoozy.bin
Invalid model file
Traceback (most recent call last):
  File "privateGPT/privateGPT.py", line 83, in <module>
    main()
  File "privateGPT/privateGPT.py", line 38, in main
    llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', n_batch=model_n_batch, callbacks=callbacks, verbose=False)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Unable to instantiate model (type=value_error)

[Saahil-exe]

@Saahil-exe, Please share folder strcuture and contends on the .env file

PERSIST_DIRECTORY=db MODEL_TYPE=GPT4All MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2 MODEL_N_CTX=1000 TARGET_SOURCE_CHUNKS=4

I upgraded to gpt4all 0.3.4 still the same issue, the model is in the right folder as well.

[Saahil-exe]

Using embedded DuckDB with persistence: data will be stored in: db Found model file at models/ggml-gpt4all-j-v1.3-groovy.bin Invalid model file ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ D:\AI\PrivateGPT\privateGPT\privateGPT.py:82 in │ │ │ │ 79 │ │ 80 │ │ 81 if name == "main": │ │ ❱ 82 │ main() │ │ 83 │ │ │ │ D:\AI\PrivateGPT\privateGPT\privateGPT.py:38 in main │ │ │ │ 35 │ │ case "LlamaCpp": │ │ 36 │ │ │ llm = LlamaCpp(model_path=model_path, n_ctx=model_n_ctx, n_batch=model_n_bat │ │ 37 │ │ case "GPT4All": │ │ ❱ 38 │ │ │ llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', n_batch=m │ │ 39 │ │ case _default: │ │ 40 │ │ │ print(f"Model {model_type} not supported!") │ │ 41 │ │ │ exit; │ │ │ │ D:\AI\PrivateGPT\privateGPT\pydantic\main.py:341 in pydantic.main.BaseModel.init │ │ │ │ [Errno 2] No such file or directory: 'D:\AI\PrivateGPT\privateGPT\pydantic\main.py' │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ ValidationError: 1 validation error for GPT4All root Unable to instantiate model (type=value_error)

[Saahil-exe]

and if i try the hermes model it says this:

validation error for LlamaCpp root Could not load Llama model from path: models/ggml-v3-13b-hermes-q5_1.bin. Received error [WinError -1073741795] Windows Error 0xc000001d (type=value_error) Exception ignored in: <function Llama.del at 0x0000000024A67EC0> Traceback (most recent call last): File "C:\Users\Saahil\AppData\Roaming\Python\Python311\site-packages\llama_cpp\llama.py", line 1076, in del if self.ctx is not None: ^^^^^^^^ AttributeError: 'Llama' object has no attribute 'ctx'

[FearTheBadger]

@Saahil-exe in your .env you are missing the MODEL_N_BATCH=8. I'm not sure if that matters or not.

I just got things to work on a new clean Ubuntu 22.04 install:

sudo apt upgrade
sudo apt install python3-pip
pip3 install --user -r requirements.txt
pip3 install --user gpt4all==0.3.3
mkdir models
cd models
wget https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin
cd ..
cp example.env .env
python3 ingest.py
python3 privateGPT.py

[Saahil-exe]

I added MODEL_N_BATCH=8 later on and it still didnt work.

[Saahil-exe]

@Saahil-exe in your .env you are missing the MODEL_N_BATCH=8. I'm not sure if that matters or not.

I just got things to work on a new clean Ubuntu 22.04 install:

sudo apt upgrade
sudo apt install python3-pip
pip3 install --user -r requirements.txt
pip3 install --user gpt4all==0.3.3
mkdir models
cd models
wget https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin
cd ..
cp example.env .env
python3 ingest.py
python3 privateGPT.py

Did you face the same error as me on windows and then it worked on ubuntu?

[boral]

Got same error in Windows

Getting following error on running python privateGPT.py

(myenv) (base) PS C:\Users\hp\Downloads\privateGPT-main> python privateGPT.py
Using embedded DuckDB with persistence: data will be stored in: db
Found model file at  models/ggml-v3-13b-hermes-q5_1.bin
Invalid model file
Traceback (most recent call last):
  File "C:\Users\hp\Downloads\privateGPT-main\privateGPT.py", line 82, in <module>
    main()
  File "C:\Users\hp\Downloads\privateGPT-main\privateGPT.py", line 38, in main
    llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', n_batch=model_n_batch, callbacks=callbacks, verbose=False)
  File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Unable to instantiate model (type=value_error)

[boral]

Updating to gpt4all 0.3.4 works. Not able to create pull requests. But someone need to update requirements.py

[andrewdavid48]

That still doesn't fix the issue when trying to load a the model on Mac M1 Max

[JamesO2x]

This is my first attempt at running a local LLM, so I'm totally new to this. But I just wanted to add my own confirmation: updating to gpt4all 0.3.4 works for me. I've got a 12 year old CPU and currently running on Windows 10. I did nothing other than follow the instructions in the ReadMe, clone the repo, and change the single line from gpt4all 0.3.2 to gpt4all 0.3.4.

Everything for me basically worked "out of the box".

So my guess is that whatever issue @boral and @andrewmurraydavid and others are having is possibly related to something else?

[Saahil-exe]

@JamesO2x 12 year old cpus can be a pain, mine is approximately that old too, can you mention the specs of your machine?

Mine is

• Intel® Core™ i5 i5-460M 2.53 GHz
• 6 GB DDR3-SDRAM 1066 MHz 1 x 2 + 1 x 4 GB
• NVIDIA® GeForce® GT 425M Intel® HD Graphics

[JamesO2x]

Mine is:

• Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz 3.10 GHz
• 16 GB RAM, DDR3 DIMM 1333 MHz
• NVIDIA GeForce GT 1030

[watchmen1992]

I upgraded to 0.3.4, but the problem still exists

OS：debian 10.3 python：3.10.10

This is the configuration of the. env file

This problem has been bothering me all afternoon

[andrewdavid48]

Yeah, what @watchmen1992 is experiencing is what I've been experiencing since the major update on gpt4all. I mentioned my OS and CPU because there can be compatibility issues, but if this is an issue on mac, windows and linux, there must be something else going on here. I've also played around with different versions of langchain without success. FYI @imartinez @FearTheBadger

[ralphbk]

Same here on Windows 10. Using gpt4all-0.3.4 (other versions gave various other problems listed above).

C:\privateGPT>python privateGPT.py Using embedded DuckDB with persistence: data will be stored in: db Traceback (most recent call last): File "C:\privateGPT\privateGPT.py", line 83, in main() File "C:\privateGPT\privateGPT.py", line 38, in main llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', n_batch=model_n_batch, callbacks=callbacks, verbose=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All root Invalid model directory: models/ (type=value_error)

.env is the unchanged from the example.

PERSIST_DIRECTORY=db MODEL_TYPE=GPT4All MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2 MODEL_N_CTX=1000 MODEL_N_BATCH=8 TARGET_SOURCE_CHUNKS=4

Is there a way to get this working?

[ralphbk]

OK. The solution for this was to download the ggml-gpt4all-j-v1.3-groovy.bin from the "Model Explorer" section https://gpt4all.io/index.html and place it in a "models" folder in the privateGPT folder.

[greggft]

I've rebuilt my VM from "scratch" followed @Saahil-exe 's commands tried all gpt4all versions 2.3 and up downloaded multiple files from the model explorer built gpt4all manually all give me Using embedded DuckDB with persistence: data will be stored in: db Found model file at /root/.cache/gpt4all/ggml-gpt4all-j-v1.3-groovy.bin Invalid model file Traceback (most recent call last): File "/root/privateGPT/privateGPT.py", line 84, in main() File "/root/privateGPT/privateGPT.py", line 39, in main llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', n_batch=model_n_batch, callbacks=callbacks, verbose=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All root Unable to instantiate model (type=value_error)

I don't have an example.env file My .env file has the right stuff

If I switch from GPT4ALL to LlamaCpp I get Using embedded DuckDB with persistence: data will be stored in: db llama.cpp: loading model from models/ggml-gpt4all-j-v1.3-groovy.bin terminate called after throwing an instance of 'std::runtime_error' what(): unexpectedly reached end of file Aborted

I would really like to get this to work.

Ubuntu 23.04

[rohith50]

After trying pip install multiple gpt4all versions eg: 0.3.4, 0.3.3 0.3.5. the error is still occurring while loading the models.

Building gpt4all from source fixed the issue for me.. The instructions to build are here : https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md

OS : Ubuntu 18.04

[greggft]

@rohith50 I already tried building gpt4all myself Still did not work