private-gpt icon indicating copy to clipboard operation
private-gpt copied to clipboard
verified publisher icon zylon-ai

Groq API in PrivateGPT

Open psmukhopadhyay opened this issue 1 year ago • 5 comments

Hello

I am a beginner-level user of PrivateGPT and set it for 'local' with LLM as mistral-7b-instruct-v0.2.Q4_K_M.gguf.

Please advise me how to add Groq (OpenAI compatible LLM service - https://console.groq.com/docs/quickstart) in PrivateGPT. I already have API key and other ID for Groq. Should I follow the included azopenai as model?

Regards

psmukhopadhyay avatar May 23 '24 06:05 psmukhopadhyay

I am in the same boat... Will let you know if I discover the solution.

KenFlack avatar May 27 '24 14:05 KenFlack

Thanks Ken :)

psmukhopadhyay avatar May 27 '24 15:05 psmukhopadhyay

any updates guys?

nizar0x1f avatar Sep 07 '24 02:09 nizar0x1f

yes i was able to integrate it with fireworks hoping it'll be similar to groq will create a PR for this

somashekhar161 avatar Sep 20 '24 07:09 somashekhar161

any updates guys?

https://github.com/zylon-ai/private-gpt/pull/2089 check this out added fireworks integration

somashekhar161 avatar Sep 20 '24 09:09 somashekhar161