ClevCode Ltd

This is such an obviously very useful PR + it's such a small and simple patch. How come it has not been merged yet?

When looking at the current code, I see that support has been added specifically for grammars conforming to JSON which are then translated into the corresponding GGBNF, and although that's...

I really hope this can finally be merged. It's really odd to me that this simple but essential feature hasn't been merged yet, after multiple pull request submissions, and that...

PS. https://www.promptingguide.ai/research/llm-tokenization As pointed out by Karpathy, due to tokenization, YAML is a much better format than JSON for producing structured output from an LLM. Just one of many reasons...

A really simple Modelfile example, to ensure that a model only answers with a Python code block :smile: FROM deepseek-coder PARAMETER grammar """ root ::= "\x60\x60\x60python3\n" [^\x60]+ "\n\x60\x60\x60" """

Theoretically it would even be possible to enforce that the actual code produced is syntactically valid Python code. Adding a good SYSTEM prompt helps a lot of course. Here's an...

Another example: ``` $ wget https://raw.githubusercontent.com/ggerganov/llama.cpp/master/examples/json-schema-to-grammar.py $ cat > movie-schema.json Modelfile

PS. The reason I'm telling deepseek-coder that it's an AI developed by OpenAI is because of this pretty hilarious result :smile: "Telling mixtral that it is "ChatGPT developed by OpenAI"...

I would suggest making a simple stand alone GBNF grammar validator based on llama.cpp sources (as a C-exported function, that can be linked and used directly by ollama with a...