llama.cpp
llama.cpp copied to clipboard
Misc. bug: Problems with official jinja templates (Gemma 2, Llama 3.2, Qwen 2.5)
Name and Version
llama-cli --version version: 4713 (a4f011e8) built with MSVC 19.42.34436.0 for x64
Operating systems
Windows
Which llama.cpp modules do you know to be affected?
llama-server
Command line
1. llama-server -ngl 99 -m gemma-2-2b-it-Q8_0.gguf --jinja --chat-template-file gemma2.jinja -c 8192
2. llama-server -ngl 99 -m Llama-3.2-3B-Instruct-Q8_0.gguf --jinja --chat-template-file llama3.2.jinja -c 8192
3. llama-server -ngl 99 -m Qwen2.5-1.5B-Instruct-Q8_0.gguf --jinja --chat-template-file qwen2.5.jinja -c 8192
Problem description & steps to reproduce
Extracting official chat templates from chat_template field in tokenizer_config.json (Gemma 2, Llama 3.2, Qwen 2.5), storing them in files, and then trying to use them with llama-server results in errors.
- Gemma 2:
parse: error parsing grammar: expecting name at
after each message. - Llama 3.2: server doesn't start.
- Qwen 2.5:
parse: error parsing grammar: expecting name at
after each message.
@ochafik Could you look into this? It would be nice to have jinja implementation fully working with official templates, at least for major models.
First Bad Commit
No response
Relevant log output