llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

[Proposal] "Stable" C API

Open Ronsor opened this issue 1 year ago • 0 comments

I propose refactoring main.cpp into a library (llama.cpp, compiled to llama.so/llama.a/whatever) and making main.cpp a simple driver program. A simple C API should be exposed to access the model, and then bindings can more easily be written for Python, node.js, or whatever other language.

This would partially solve #82 and #162.

Ronsor avatar Mar 15 '23 18:03 Ronsor