camel icon indicating copy to clipboard operation
camel copied to clipboard

[Feature] Add `llama.cpp` local backend model support

Open dandansamax opened this issue 1 year ago • 1 comments

Description

Using OpenAI models requires connections to international Internet and OpenAI api key, which may be difficult for some users. Introducing local model backend such as llama.cpp, which integrates large number of opensource model and hardware supports (Apple M-chips, CUDA, CPU, etc.) can be a good solution.

See llama.cpp.

dandansamax avatar Dec 04 '23 11:12 dandansamax

@ocss884 will be working on this.

lightaime avatar Dec 04 '23 13:12 lightaime