Error while runing GO Code

[harjitdotsingh]

System Info

Iahve compiled the go code and all the bindings when I run the code with the following arguments

./main -m /home/azureuser/models/ggml-gpt4all-j/ggml-gpt4all-j.bin -t 2

I also tried with ./main -m /home/azureuser/models/ggml-gpt4all-j -t 2

I Get the following error

load_model: error 'Invalid argument' Loading the model failed: failed loading model

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] backend
• [X] bindings
• [ ] python-bindings
• [ ] chat-ui
• [ ] models
• [ ] circleci
• [ ] docker
• [ ] api

Reproduction

NA

Expected behavior

Model should be loaded

[cosmic-snow]

I've just tried to help resolve a problem with the Python bindings on Ubuntu 20.04. I've never attempted to use the Go bindings, but maybe the solution in your case is similar.

Recently there was a change in how it deals with the underlying llama.cpp project. If the Go bindings haven't been fixed to work with that yet, and if you're building yourself from the repo, you could try checking out the commit: f4a1f7340c76e7d925035fda89315aaeb69f09f0 or something earlier. The mentioned one is right before that change was introduced.

Sorry, but I'm not familiar with Go, so I'm not going to try that myself.

[ekle]

I have the same problem. https://github.com/nomic-ai/gpt4all/commit/f4a1f7340c76e7d925035fda89315aaeb69f09f0 doesn't work with most models. So var it only works with ggml-gpt4all-j-v1.3-groovy.bin for me.

[harjitdotsingh]

@cosmic-snow - It's an issue with the version of llama.cpp. Switching to specific version you mentioned works for me. I also went and got the latest code of llama and tried running the models but the issue is the same. There it was mentioned that the models have to be retrained to match the version

[cosmic-snow]

I have the same problem. https://github.com/nomic-ai/gpt4all/commit/f4a1f7340c76e7d925035fda89315aaeb69f09f0 doesn't work with most models.

Strange. It should work with all downloadable models on offer so far.

It's an issue with the version of llama.cpp. Switching to specific version you mentioned works for me. I also went and got the latest code of llama and tried running the models but the issue is the same. There it was mentioned that the models have to be retrained to match the version

Looks like the Go bindings not being fixed yet was confirmed here: https://github.com/nomic-ai/gpt4all/issues/826#issuecomment-1575754383.

You won't have to retrain any old models after everything is fixed. The underlying llama.cpp is now versioned on the main branch, which means it should be able work with old and new versions of the model formats alike. But the API had to be changed for that, so just wait for the bindings to get fixed.

[MonkeyRDluffy]

I still have the issue , is it supposed to be fixed ?