continue icon indicating copy to clipboard operation
continue copied to clipboard

"useLegacyCompletionsEndpoint": false is not respected for Azure provider

Open ukw2d opened this issue 1 year ago • 0 comments

Before submitting your bug report

Relevant environment info

- OS: macOS Sonoma 14.5
- Continue: v0.9.198
- IDE: VSCode 1.92.2 (Universal)
- Model: Any AzureOpenAI model
- config.json:
  {
  "models": [
    {
      "title": "Anthropic Claude V3.5 Sonnet",
      "provider": "azure",
      "model": "claude-3-sonnet-20240229",
      "apiBase": "https://ai-proxy.lab.---.com",
      "engine": "anthropic.claude-v3-5-sonnet",
      "apiVersion": "2023-07-01-preview",
      "apiType": "azure",
      "apiKey": "---"
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Code LLaMA 34B Instruct",
    "provider": "azure",
    "model": "codellama-34b",
    "apiBase": "https://ai-proxy.lab.---.com",
    "engine": "CodeLlama-34b-Instruct-hf",
    "apiVersion": "2023-07-01-preview",
    "apiType": "azure",
    "apiKey": "---",
    "useLegacyCompletionsEndpoint": false
  },
  "tabAutocompleteOptions": {
    "useCopyBuffer": true,
    "maxPromptTokens": 400,
    "prefixPercentage": 0.5
  },
  "contextProviders": [
    {
      "name": "code",
      "params": {}
    },
    {
      "name": "docs",
      "params": {}
    },
    {
      "name": "diff",
      "params": {}
    },
    {
      "name": "terminal",
      "params": {}
    },
    {
      "name": "problems",
      "params": {}
    },
    {
      "name": "folder",
      "params": {}
    },
    {
      "name": "codebase",
      "params": {}
    }
  ],
  "slashCommands": [
    {
      "name": "edit",
      "description": "Edit selected code"
    },
    {
      "name": "comment",
      "description": "Write comments for the selected code"
    },
    {
      "name": "share",
      "description": "Export the current chat session to markdown"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command"
    },
    {
      "name": "commit",
      "description": "Generate a git commit message"
    }
  ],
  "embeddingsProvider": {
    "title": "Text-Embedding-3-Large",
    "provider": "openai",
    "model": "text-embedding-3-large",
    "apiBase": "https://ai-proxy.lab.---.com/",
    "engine": "text-embedding-3-large-1",
    "apiVersion": "2023-07-01-preview",
    "apiType": "azure",
    "apiKey": "---"
    }
}



### Description

"useLegacyCompletionsEndpoint": false is not respected on the latest preview release of Continue at least for Azure provider/api type. As per attached config.json, I see the error:

`workbench.desktop.main.js:sourcemap:820 HTTP 404 Not Found from https://ai-proxy.lab.epam.com/openai/deployments/CodeLlama-34b-Instruct-hf/completions?api-version=2023-07-01-preview  This may mean that you forgot to add '/v1' to the end of your 'apiBase' in config.json.`


Which says that despite having negative useLegacyCompletionsEndpoint flag, url path is not changed to /chat/completions.

P.S. Interesting observation: using gpt-* models for "tabAutocompleteOptions" applies /chat/completions automatically and autocomplete actually works even without useLegacyCompletionsEndpoint flag if I use corresponded engine from azure provider. But If I try to use other engines/models than gpt-* - I encounter the issue described above.

### To reproduce

1. Set up non-gpt model & engine in "tabAutocompleteModel" using azure provider and api type
2. Change "tabAutocompleteModel" with "useLegacyCompletionsEndpoint": false
3. Try autocomplete and see 404 error

### Log output

```Shell
console.ts:137 [Extension Host] HTTP 404 Not Found from https://ai-proxy.lab.epam.com/openai/deployments/CodeLlama-34b-Instruct-hf/completions?api-version=2023-07-01-preview

This may mean that you forgot to add '/v1' to the end of your 'apiBase' in config.json.

Code: undefined
Error number: undefined
Syscall: undefined
Type: undefined

Error: HTTP 404 Not Found from https://ai-proxy.lab.epam.com/openai/deployments/CodeLlama-34b-Instruct-hf/completions?api-version=2023-07-01-preview

This may mean that you forgot to add '/v1' to the end of your 'apiBase' in config.json.
	at customFetch (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:374458:21)
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
	at async withExponentialBackoff (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:373232:26)
	at async Azure._legacystreamComplete (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:375137:26)
	at async Azure._streamChat (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:375153:28)
	at async Azure._streamComplete (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:375126:26)
	at async Azure.streamComplete (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:374580:26)
	at async ListenableGenerator._start (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:96231:28)
console.ts:137 [Extension Host] Error generating autocompletion:  Error: HTTP 404 Not Found from https://ai-proxy.lab.epam.com/openai/deployments/CodeLlama-34b-Instruct-hf/completions?api-version=2023-07-01-preview

This may mean that you forgot to add '/v1' to the end of your 'apiBase' in config.json.
	at customFetch (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:374483:19)
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
	at async withExponentialBackoff (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:373232:26)
	at async Azure._legacystreamComplete (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:375137:26)
	at async Azure._streamChat (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:375153:28)
	at async Azure._streamComplete (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:375126:26)
	at async Azure.streamComplete (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:374580:26)
	at async ListenableGenerator._start (/Users/Vladyslav_Andriyev/.vscode/extensions/continue.continue-0.9.198-darwin-x64/out/extension.js:96231:28)
notificationsAlerts.ts:42 HTTP 404 Not Found from https://ai-proxy.lab.epam.com/openai/deployments/CodeLlama-34b-Instruct-hf/completions?api-version=2023-07-01-preview  This may mean that you forgot to add '/v1' to the end of your 'apiBase' in config.json.

ukw2d avatar Aug 22 '24 07:08 ukw2d