llama-api-server icon indicating copy to clipboard operation
llama-api-server copied to clipboard

Published 20 hours ago • verified publisher icon iaalm

[FEATURE] Upgrade llama-cpp-python to latest version

Open RomainMendez opened this issue 2 years ago • 2 comments

The version of llama-cpp-python this project uses is quite old. Therefore I get a lot of errors regarding versions of GGML models. It also doesn't support GGUF models.

I would suggest to up the version of llama-cpp-python to the latest one.

GGUF models are the future anyway it would seem. Perhaps we could work on something that allows chosing ?

RomainMendez avatar Sep 23 '23 12:09 RomainMendez

Thanks for remaindering.👍

iaalm avatar Sep 26 '23 01:09 iaalm

This project actually depends on llamacpp-python. Seems their most recent version doesn't support gguf. I will see whether they can have an update.

iaalm avatar Oct 15 '23 03:10 iaalm