Groq api uses incorrect apibase on refactor mode (cmd+I)

[Altroo]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [x] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Windows 11
- Continue: 0.0.45
- IDE: Pycharm 2024.1

Description

Alright so, this is my first time using this plugin, it is really interesting.

i've configured Groq using an api key with the model llama3-70b : config.json :

{
    "model": "Llama3-70b-8192",
    "title": "Groq : Llama3-70b-8192",
    "apiKey": "key",
    "provider": "groq"
}

It's working on the "continue" tab, however when using refactor mode with "cmd + I", it raises an error as shown in core.log :

Error running handler for "command/run":  Error: HTTP 404 Not Found from https://api.groq.com/openai/v1/completions
{"error":{"message":"Unknown request URL: POST /openai/v1/completions. Please check the URL for typos, or see the docs at https://console.groq.com/docs/","type":"invalid_request_error","code":"unknown_url"}}
Error: HTTP 404 Not Found from https://api.groq.com/openai/v1/completions

Adding the correct "apiBase" as "https://api.groq.com/openai/v1/chat" fixes it but raises another error :

Error running handler for "streamDiffLines":  Error: HTTP 400 Bad Request from https://api.groq.com/openai/v1/chat/completions
{"error":{"message":"property 'prompt' is unsupported, did you mean 'stop'?","type":"invalid_request_error"}}
Error: HTTP 400 Bad Request from https://api.groq.com/openai/v1/chat/completions

[EricBizet]

I can confirm I am experiencing the same issue in VS Code/Ubuntu23.10, thank you for opening this ticket

[sestinj]

@Altroo @EricBizet Just fixed this here. We were detecting from the OpenAI class but forgot to check when we made Groq it's own LLM provider.

Until this is released later today, you can use "provider": "openai" with "apiBase": "https://api.groq.com/openai/v1/" instead and it should work

Also working with Groq to hopefully add support for the completions endpoint : )