continue icon indicating copy to clipboard operation
continue copied to clipboard

Using a tab completions template (moustache) encodes special characters to html

Open Lucki opened this issue 11 months ago • 0 comments

Before submitting your bug report

Relevant environment info

- OS: Arch Linux
- Continue: v0.9.80 (pre-release)
- IDE: VSCode 1.87.0

Description

Defining a template in tabAutocompleteOptions will html encode special characters before they're sent to the model. E.g.: " becomes ", ' becomes ' - The model adapts to this and the response is equally useless.

To reproduce

  1. Define a template in tabAutocompleteOptions. E.g. for deepseek-coder:
    "tabAutocompleteOptions": {
      "template": "<|fim▁begin|>{{prefix}}<|fim▁hole|>{{suffix}}<|fim▁end|>"
    },
    
  2. Create a new file, make sure autocomplete is turned on.
  3. Write something containing quotation marks. E.g.: print("asd")
  4. Press Return and enter print: grafik
  5. View the log of your LLM server. Example from Koboldcpp:
    {
        "model": "AUTODETECT",
        "max_tokens": 1024,
        "temperature": 0,
        "stop": [
            "\n\n",
            "```",
            "function",
            "class",
            "module",
            "export"
        ],
        "prompt": "<\uff5cfim\u2581begin\uff5c>print(&quot;asd&quot;)\np<\uff5cfim\u2581hole\uff5c><\uff5cfim\u2581end\uff5c>",
        "stream": true
    }
    

Log output

Notice &quot; vs \" in both examples as a replacement for ".

Log with template:

{
    "model": "AUTODETECT",
    "max_tokens": 1024,
    "temperature": 0,
    "stop": [
        "\n\n",
        "",
        "function",
        "class",
        "module",
        "export"
    ],
    "prompt": "<\uff5cfim\u2581begin\uff5c>print(&quot;asd&quot;)\np<\uff5cfim\u2581hole\uff5c><\uff5cfim\u2581end\uff5c>",
    "stream": true
}

Log without template: (this doesn't acutally give me any output because it's the wrong format)

{
    "model": "AUTODETECT",
    "max_tokens": 1024,
    "temperature": 0,
    "stop": [
        "<fim_prefix>",
        "<fim_suffix>",
        "<fim_middle>",
        "<|endoftext|>",
        "\n\n",
        "```",
        "function",
        "class",
        "module",
        "export"
    ],
    "prompt": "<fim_prefix>// Path: config.json\n// ]\n// }\n// \n//   \"slashCommands\": [\n//     {\n//       \"name\": \"edit\",\n//       \"description\": \"Edit selected code\"\n//\n\nprint(\"asd\")\nprint<fim_suffix><fim_middle>",
    "stream": true
}

tabAutocompleteModel config:

"tabAutocompleteModel": {
  "title": "asd",
  "provider": "openai",
  "model": "AUTODETECT",
  "apiBase": "http://localhost:5001/v1",
  "template": "deepseek",
  "apiKey": ""
},

Lucki avatar Mar 06 '24 19:03 Lucki