how do i change the models: llm or embedding

[cleesmith]

It seems that editing settings.yaml and running again isn't enough to change to different models.

Are there any instructions for exactly how to do this? I see nothing in the docs.

I'm ok if errors happen or the new models don't work properly.

Also, is it possible to use LLM Studio which provides OpenAI-like endpoint, as a way to more easily test other model?

Otherwise, this works well ... Thank you!

[icsy7867]

Models have to be downloaded.

If you set the tokenizer model, which llm you are using and the file name, run scripts/setup and it will automatically grab the corresponding models.

[Apotrox]

So far i changed the models (LLM and embedding) in the settings.yaml and settings-docker.yaml, and after running docker compose run --rm --entrypoint="/usr/bin/env python3 scripts/setup" private-gpt it seems to at least download them. it's still at it so i can't say for sure if it worked or not, will keep this updated. Update: Worked, the Docker app is starting with the custom LLM and embedding

[JanuRam]

Was facing the same issue. The above solution solved it. Thank you @Apotrox

[Walker555]

I wanted to use a German speaking model. Using Ubuntu with pyenv.

step 1- create settings-de.yaml

local:
  llm_hf_repo_id: TheBloke/em_german_leo_mistral-GGUF
  llm_hf_model_file:   em_german_leo_mistral.Q4_K_M.gguf
  embedding_hf_model_name: T-Systems-onsite/german-roberta-sentence-transformer-v2
  #llama, default or tag
  prompt_style: "default"

step 2 - download the model from huggingface hub with the setup script. It will keep all already downloaded files in models subdir.

PGPT_PROFILES=de ./scripts/setup

step 3 - run the app

PGPT_PROFILES=de python3.11 -m private_gpt

[hamvi-ai]

is it possible to use LLM Studio which provides OpenAI-like endpoint, as a way to more easily test other model?

[icsy7867]

Check out the openailike mode.

https://docs.privategpt.dev/manual/advanced-setup/llm-backends

[cleesmith]

Thanks

[robzilla1738]

I went into the settings-ollama.yaml and changed the name of the model there from Mistral to any other llama model. Whe nI restarted the Private GPT server it loaded the one I changed it to.

[terrornoize]

I went into the settings-ollama.yaml and changed the name of the model there from Mistral to any other llama model. Whe nI restarted the Private GPT server it loaded the one I changed it to.

And directly download the model only with parameter change in the yaml file? Does the new model also maintain the possibility of ingesting personal documents?