gpt4all-ui load model onto gpu not cpu

[gbrackenridge]

Expected Behavior

Ability to invoke ggml model in gpu mode using gpt4all-ui

Current Behavior

Unclear how to pass the parameters or which file to modify to use gpu model calls.

Steps to Reproduce

Install gpt4all-ui run app.py model loaded via cpu only

Possible Solution

Pass the gpu parameters to the script or edit underlying conf files (which ones?)

Context

Have gp4all running nicely with the ggml model via gpu on linux/gpu server. Trying to use the fantastic gpt4all-ui application. Struggling to figure out how to have the ui app invoke the model onto the server gpu. It is stunningly slow on cpu based loading. Many many thanks to any advice providers!

[ParisNeo]

I don't think that the pyllamacpp supports gpu yet.

[andzejsp]

I don't think that the pyllamacpp supports gpu yet.

On the GPT4all repo there is this https://github.com/nomic-ai/gpt4all#gpu-interface

Not sure how it can be compatible with this repo.

[ParisNeo]

That is not the same code. The ui uses pyllamacpp backend (that's why you need to convert your model before starting).

As far as I know, this backend does not yet support gpu (or at least the python binding doesn't allow it yet).

It will eventually be possible to force Using GPU, and I'll add it as a parameter to the configuration file.