private-gpt
private-gpt copied to clipboard
zylon-ai
Upgrade llamacpp python to 0.1.53
Upgrading llamacpp python to 0.1.53 would add support for ggml v3 models. Everything else is still compatible,
As I know, the required llama-cpp-python package has no version limitations, the version provided in requirements.txt is just a number...
Checkout this discussion, and I have been able to run most of the models, which were not being run till now.
https://huggingface.co/TheBloke/MPT-7B-Instruct-GGML/discussions/2
llama-cpp-python==0.1.53 ctransformers==0.2.0
Checkout this discussion, and I have been able to run most of the models, which were not being run till now.
https://huggingface.co/TheBloke/MPT-7B-Instruct-GGML/discussions/2
llama-cpp-python==0.1.53 ctransformers==0.2.0
Were you able to get MPT-7B running with privateGPT?
As I know, the required llama-cpp-python package has no version limitations, the version provided in requirements.txt is just a number...
Yes, I've upgraded locally without any issues. I think it would be worth changing the version in the requirements.txt for ease of use though. The PR would be a simple matter of changing the version number in the requirements.txt file. I can do it when I get the time.