chat-ui icon indicating copy to clipboard operation
chat-ui copied to clipboard

Cannot "Download prompt and parameters" for DeepSeek-R1-Distill-Qwen-32B

Open evalstate opened this issue 1 year ago • 0 comments

Bug description

500 Server Error raised when navigating to /conversation/[id]/message/[id]/prompt for DeepSeek-R1-Distill-Qwen-32B Chats.

Steps to reproduce

Use the "Download prompt and parameters button" on a message in a DS-R1 conversation.

Screenshots

Context

Happens on chatui, reproduced locally.

Logs

The issue arises from calling the @huggingface/jinja constructor with the below template.

 error: {
      "stack":
          Error: Parser Error: Expected closing statement token. OpenSquareBracket !== CloseStatement.
              at expect (..../node_modules/@huggingface/transformers/dist/transformers.mjs:537:13)
              at parseJinjaStatement (...../node_modules/@huggingface/transformers/dist/transformers.mjs:569:9)
              at parseAny (...../node_modules/@huggingface/transformers/dist/transformers.mjs:546:16)
              at parseIfStatement (...../node_modules/@huggingface/transformers/dist/transformers.mjs:618:17)
              at parseJinjaStatement (...../node_modules/@huggingface/transformers/dist/transformers.mjs:573:18)

Specs

Config

Notes

This is the chat template returned that causes the error:

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<|User|>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<|Assistant|><|tool▁calls▁begin|><|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}{{'<|tool▁calls▁end|><|end▁of▁sentence|>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<|tool▁outputs▁end|>' + message['content'] + '<|end▁of▁sentence|>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<|Assistant|>' + content + '<|end▁of▁sentence|>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<|tool▁outputs▁begin|><|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\\n<|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<|tool▁outputs▁end|>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<|Assistant|>'}}{% endif %}

evalstate avatar Jan 26 '25 18:01 evalstate