Gemma 3 support

[igorschlum]

System Info

I installed GPT4All, opened it, downloaded the Gemma3 Instruct for hugging face (tried two models

https://huggingface.co/Mungert/gemma-3-12b-it-gguf https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF

Encountered an error loading model:

"Unsupported model architecture gemma3"

Model loading failures can happen for a variety of reasons, but the most common causes include a bad file format, an incomplete or corrupted download, the wrong file type, not enough system RAM or an incompatible model type. Here are some suggestions for resolving the problem:

• Ensure the model file has a compatible format and type

• Check the model file is complete in the download folder
• You can find the download folder in the settings dialog

• If you've sideloaded the model ensure the file is not corrupt by checking mdSsum + Read more about what models are supported in our documentation for the gui

• Check out our discord channel for help I didn't do anything custom, so I'm not sure why I've got the error.

Gpt4All version: 3.10.0 Platform: MacOS

Expected behavior

Expected to run model without getting error ("Expected" is a strong word - I'm grateful for anything at all 😄!)

[Marrim]

It seems that GPT4All had no llama.cpp backend updates for a while. Therefore some newer models with newer architectures (Phi4-mini, Gemma 3, Falcon 3) are unable to load. It may not be as trivial as pulling a fresher version of llama.cpp from upstream and enabling the new arch in the whitelist, because GPT4All is (likely) using a modified version of llama.cpp.

[paulocoutinhox]

Getting same error: "untimeError: Unable to instantiate model: Unsupported model architecture: gemma3".

There is some Python library that works like gpt4all and has llama.cpp updated?

[adhdboy411]

Well there's always LMStudio. I'm personally waiting for Jan AI to get updated, but ah well.

[Titaniumtown]

I wouldn't use any non-free software LLM inference software @adhdboy411

[TheAlex25]

Yeah, it feels like GPT4All is falling behind other front-ends, as GPT4All has not implemented multimodal models such as Mistral Small 3.1 either. The back end, Llama.cpp, needs to catch up. I will try to stick with GPT4All, as it is open-source and has built-in RAG.

[igorschlum]

@TheAlex25 I really think that GPT-4 is a good solution for beginners. I was wondering if some tried to use Ollama as an external IA provider and still use local folders for RAG.

[TheAlex25]

Maybe, but I do not want to resort to Ollama, unless GPT4All implements Ollama as a back end. It might be time for GPT4All to retire Llama.cpp and switch to Ollama as a back end. GPT4All works out of the box, and I want to keep it as long as possible.

[paulocoutinhox]

Hi,

If GPT4ALL update llama.cpp i think that most problems was solved.

Why the author don't update?

Thanks.

[adhdboy411]

I concur (hehe) with @paulocoutinhox. LMStudio is based of Llama.cpp (hehe llama.ccp would be funny), and that works perfectly. Same with Jan, which is open source but is updated slower than LMStudio and has far less features, although it has some really useful features.

[Seedmanc]

Jan is terrible, Msty seems promising, but they lack clear indication of RAG errors.