llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

Misc. bug: Problems with official jinja templates (Gemma 2, Llama 3.2, Qwen 2.5)

Open MoonRide303 opened this issue 1 week ago • 11 comments

Name and Version

llama-cli --version version: 4713 (a4f011e8) built with MSVC 19.42.34436.0 for x64

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

llama-server

Command line

1. llama-server -ngl 99 -m gemma-2-2b-it-Q8_0.gguf --jinja --chat-template-file gemma2.jinja -c 8192
2. llama-server -ngl 99 -m Llama-3.2-3B-Instruct-Q8_0.gguf --jinja --chat-template-file llama3.2.jinja -c 8192
3. llama-server -ngl 99 -m Qwen2.5-1.5B-Instruct-Q8_0.gguf --jinja --chat-template-file qwen2.5.jinja -c 8192

Problem description & steps to reproduce

Extracting official chat templates from chat_template field in tokenizer_config.json (Gemma 2, Llama 3.2, Qwen 2.5), storing them in files, and then trying to use them with llama-server results in errors.

  1. Gemma 2: parse: error parsing grammar: expecting name at after each message.
  2. Llama 3.2: server doesn't start.
  3. Qwen 2.5: parse: error parsing grammar: expecting name at after each message.

@ochafik Could you look into this? It would be nice to have jinja implementation fully working with official templates, at least for major models.

First Bad Commit

No response

Relevant log output


MoonRide303 avatar Feb 14 '25 09:02 MoonRide303