ollama icon indicating copy to clipboard operation
ollama copied to clipboard

Mixtral 8x22b template update

Open slavonnet opened this issue 9 months ago • 0 comments

If we look here https://ollama.com/library/mixtral:8x22b/blobs/c43332387573 . template is [INST] {{ if .System }}{{ .System }} {{ end }}{{ .Prompt }} [/INST]

if we look here https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1/commit/95d063951382d47385fe7b36e202b68639e5c066 template have more options

"template": "{{bos_token}}{% set user_messages = messages | selectattr('role', 'equalto', 'user') | list %}{% for message in messages %}{% if message['role'] == 'user' %}{% if message == user_messages[-1] %}{% if tools %}{{'[AVAILABLE_TOOLS]'+ tools|string + '[/AVAILABLE_TOOLS]'}}{% endif %}{{ '[INST]' + message['content'] + '[/INST]' }}{% else %}{{ '[INST]' + message['content'] + '[/INST]' }}{% endif %}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' ' + eos_token}}{% elif message['role'] == 'tool_results' %}{{'[TOOL_RESULTS]' + message['content']|string + '[/TOOL_RESULTS]'}}{% elif message['role'] == 'tool_calls' %}{{'[TOOL_CALLS]' + message['content']|string + eos_token}}{% endif %}{% endfor %}"

In description https://ollama.com/library/mixtral:8x22b say It is natively capable of function calling but in template no have support it. Look like need add TOOL_RESULTS, TOOL_CALLS, AVAILABLE_TOOLS

Maybe need some updates?

slavonnet avatar May 07 '24 13:05 slavonnet