marsha icon indicating copy to clipboard operation
marsha copied to clipboard

Use llama.cpp with Python grammar syntax

Open AlphaAtlas opened this issue 1 year ago • 1 comments

llama.cpp just implemented a feature that forces the llm to conform to arbitrary grammar, like, say, Python:

https://github.com/ggerganov/llama.cpp/pull/1773

This seems relevant for this repo. Maybe the grammar could even be swapped/generated programatically for more specific responses?

A language specific finetune of llamav2 also seems like a good idea, but perhaps that is something for another feature request...

AlphaAtlas avatar Jul 26 '23 02:07 AlphaAtlas