quivr
quivr copied to clipboard
QuivrHQ
Consider supporting custom AI models？
Support for custom models to guarantee data security issues at privatized deployment time
I manually modified the code (I haven't forked or committed yet) and managed to make it work with llama.cpp, I used the gpt4all 13b model.
It lends itself well to this kind of activity.
Unfortunately I have to say that the performance is not good, the model is not able to respond adequately as gpt3 would, and to receive an answer it takes a long time. Despite my computer having an rtx 2060 and 16 cores of ryzen 7 4800h. As soon as I have time I will upload a fork of the repository with llama support.
But don't have much hope that it will work well. I think maybe a 66B model could go a little better
Hi DanielusG, I don't have GPT API, can you share with me how can I use the gpt4all 13b model? I have try privateGPT, but not working very well, I would like to try using our quivr solution for my personal KM. Thank you very much.
BR Mark
