Can't get Mixtral to work with web-search

[iChristGit]

I have been following this project for a while and recently tried setting up oobabooga Mixtral-8x7b

I used the official prompt template used in huggingface.co :

<s> {{#each messages}}{{#ifUser}}[INST]{{#if @first}}{{#if @root.preprompt}}{{@root.preprompt}}\n{{/if}}{{/if}} {{content}} [/INST]{{/ifUser}}{{#ifAssistant}} {{content}}</s> {{/ifAssistant}}{{/each}}

Normal chat works, and summarization for the title works, but web-search does not. It always gives the full answer instead of a search term.

Here is my local.env:

MONGODB_URL=mongodb://localhost:27017
USE_LOCAL_WEBSEARCH=true
PUBLIC_APP_ASSETS=chatui
HF_ACCESS_TOKEN=hf_none
PUBLIC_APP_DESCRIPTION="ChatGPT But Open Source!"
PUBLIC_APP_NAME=ChatGPT
MODELS=`[
  {
      "name": "LocalGPT",
      "description": "Mixtral is a great overall model",
      "chatPromptTemplate" : "<s> {{#each messages}}{{#ifUser}}[INST]{{#if @first}}{{#if @root.preprompt}}{{@root.preprompt}}\n{{/if}}{{/if}} {{content}} [/INST]{{/ifUser}}{{#ifAssistant}} {{content}}</s> {{/ifAssistant}}{{/each}}",
       "preprompt": "",
       "promptExamples": [
      {
        "title": "Write an email from bullet list",
        "prompt": "As a restaurant owner, write a professional email to the supplier to get these products every week: \n\n- Wine (x10)\n- Eggs (x24)\n- Bread (x12)"
      }, {
        "title": "Code a snake game",
        "prompt": "Code a basic snake game in python and give explanations for each step."
      }, {
        "title": "Assist in a task",
        "prompt": "How do I make a delicious lemon cheesecake?"
      }
      ],
      "parameters": {
        "temperature": 0.3,
        "top_p": 0.95,
        "repetition_penalty": 1.2,
        "top_k": 50,
        "truncate": 3072,
        "max_new_tokens": 2048,
        "stop": ["</s>"]
    },
    "endpoints": [{
      "type" : "openai",
      "baseURL": "http://127.0.0.1:5000/v1"
    }]
  }
]`

[iChristGit]

@mishig25 Hey! Any idea what am I doing wrong?

[iChristGit]

@nsarrazin Can you help me figure it out?