marsha
marsha copied to clipboard
Use llama.cpp with Python grammar syntax
llama.cpp just implemented a feature that forces the llm to conform to arbitrary grammar, like, say, Python:
https://github.com/ggerganov/llama.cpp/pull/1773
This seems relevant for this repo. Maybe the grammar could even be swapped/generated programatically for more specific responses?
A language specific finetune of llamav2 also seems like a good idea, but perhaps that is something for another feature request...