AGiXT icon indicating copy to clipboard operation
AGiXT copied to clipboard

Published 20 hours ago • verified publisher icon Josh-XT

LlamaCCP Models Nonfunctional

Open gitbute opened this issue 1 year ago • 3 comments

Hi there

I tried three different models (vicuna, gpt4all and gpt4all-j), all are failing to load. The vicuna one with some strange tensor error, and the gpt4all ones he wants to convert. Do you have a working config to use a llamacpp model?

Thanks

gitbute avatar Apr 26 '23 04:04 gitbute

Hello! Our llamacpp integration is the gpt4all python module. Any model that directly works in gpt4all should also directly work in Agent-LLM.

Josh-XT avatar Apr 26 '23 14:04 Josh-XT

You can use ggml-gpt4all-l13b-snoozy. I don't think it will work but the model loads.

mtthw-meyer avatar Apr 26 '23 20:04 mtthw-meyer

Can we please see "some strange tensor error" and the prompt to convert, as well as your Llama related env settings?

Are you running in a local, remote or containerized environment?

JamesonRGrieve avatar Apr 27 '23 03:04 JamesonRGrieve