How do I format the ChatPromptTemplate ?

[iChristGit]

I currently have a working setup with llamacpp+mistral 7b instruct with the following loca.env :

MODELS=`[
  {
      "name": "Mistral",
      "chatPromptTemplate": "<s>{{#each messages}}{{#ifUser}}[INST] {{#if @first}}{{#if @root.preprompt}}{{@root.preprompt}}\n{{/if}}{{/if}} {{content}} [/INST]{{/ifUser}}{{#ifAssistant}}{{content}}</s> {{/ifAssistant}}{{/each}}",
      "parameters": {
        "temperature": 0.1,
        "top_p": 0.95,
        "repetition_penalty": 1.2,
        "top_k": 50,
        "truncate": 4096,
        "max_new_tokens": 4096,
        "stop": ["</s>"]
      },
      "endpoints": [{
         "url": "http://127.0.0.1:8080",
         "type": "llamacpp"
        }
/
      ]
  }
]`

I am trying to set up the model "Neural Chat" by intel , and the tamplate is:

System:

{system_message}

User:

{prompt}

Assistant:

How can I set the chatPromptTemplate to match it? and so it knows to summarize and search the web correctly? Im having some issues to understand how to format it, and where to put ### User ETC.

Thanks

[nsarrazin]

Hi, could you comment the template in code blocks

like this

So I can have a look ? In the meanwhile feel free to look at some of the prompt template examples we currently have to see if there's something close to what you need. I think Falcon is pretty close to what you need.

[iChristGit]

Hi, could you comment the template in code blocks

like this

So I can have a look ? In the meanwhile feel free to look at some of the prompt template examples we currently have to see if there's something close to what you need. I think Falcon is pretty close to what you need.

This is the template

### System:
{system_message}

### User:
{prompt}

### Assistant:

Ive tried to edit from the examples but still its not formatted correctly

[nsarrazin]

I think you want

### System:\n{{preprompt}}\n\n### User:{{#each messages}}{{#ifUser}}{{content}}\n\n### Assistant:{{/ifUser}}{{#ifAssistant}}{{content}}\n\n### User:{{/ifAssistant}}{{/each}}

Although I haven't tried it. Make sure you add the correct stop sequence as well in the parameters. I think in your case that should be maybe \n\n### User: or just \n\n

[iChristGit]

I think you want

### System:\n{{preprompt}}\n\n### User:{{#each messages}}{{#ifUser}}{{content}}\n\n### Assistant:{{/ifUser}}{{#ifAssistant}}{{content}}\n\n### User:{{/ifAssistant}}{{/each}}

Although I haven't tried it. Make sure you add the correct stop sequence as well in the parameters. I think in your case that should be maybe \n\n### User: or just \n\n

Thank you, will try it soon

[kindofausername]

@iChristGit Did it work?