InferLLM icon indicating copy to clipboard operation
InferLLM copied to clipboard

没有支持最新的llama.cpp的格式吗

Open AceyKubbo opened this issue 1 year ago • 0 comments

我使用llama.cpp量化过的alpaca7b_plus,看样子并没有支持

llama_model_load_internal: format     = ggjt v3 (latest)
main: seed = 1686927440
Assert ' 0 ' failed at file : /mnt/e/pyCode/InferLLM/src/graph/llama.cpp
line 37 : virtual void inferllm::LlamaGraph::load(std::shared_ptr<inferllm::InputFile>, inferllm::LlmParams&, std::shared_ptr<inferllm::Vocab>),
extra message: unsupported model type.Aborted

AceyKubbo avatar Jun 16 '23 15:06 AceyKubbo