vim_ai_complete_default and vim_ai_edit_default model can't be changed

[eadadi]

When changing the model of either vim_ai_complete_default or vim_ai_edit_default to another one (e.g. gpt-4o) then they stop working, and an error suggesting that API key should be verified is logged.

[Konfekt]

Maybe you ran into https://github.com/madox2/vim-ai/commit/ccf981974232636d08a48094bf9dad3812a43cc8#comments as well and have to set "endpoint_url" explicitly?

[eadadi]

It haven't seen that thread before, but actually, I had "endpoint_ur;" : "https://api.openai.com/v1/completions" which I changed by myself to "endpoint_url" : "https://api.openai.com/v1/chat/completions"

As I have seen that works for vim_ai_chat_default, but nope, it did not resolve the issue, still with http400 error

[Konfekt]

What model is gpt-4o? Maybe you could try with more common ones first, say gpt-4-turbo

[eadadi]

It's the one they released recently, but gpt-4-turbo does not work for me as well

[Konfekt]

Are you sure the API key is correct? That's been usually the cause of these 400 error messages if I recall correctly.

[eadadi]

Yes I do, use vim-ai a lot- just wanted to upgrade to the new model.

[Konfekt]

Maybe setting g:vim_ai_debug = 1 will reveal more than a mere 400 error.

[pedrohgmacedo]

The completions api uses a different format for the request than chat, and it does not support models later than gpt 3.5

[eadadi]

Interesting, I have seen that the 3.5-turbo-instruction called so (...-instruction) only for legacy reason, and it is not truly optimized for instruction, so seemingly we could profit from adjusting to the other models.

Maybe I will look further to suggest how to support it

[pedrohgmacedo]

Ideally, I think the plugin should be decoupled from the openai api format, and endpoint would be plugged using adapters.

[madox2]

gpt-4o model can be used only with chat engine (api /v1/chat/completions), you can configure chat engine also for completions and edits. Since /v1/completions is now considered a legacy, it could make sense to use chat engine for completions by default.

@pedrohgmacedo I like the idea of adapters, but since there are no big differences between API/model providers, I find it simpler to stick to OpenAI-compatible APIs as there are many tools/services providing such compatibility layer.

[pedrohgmacedo]

@pedrohgmacedo I like the idea of adapters, but since there are no big differences between API/model providers, I find it simpler to stick to OpenAI-compatible APIs as there are many tools/services providing such compatibility layer.

Some providers are icky and don't offer a compatible api, for instance google gemini or cohere. I'm currently hosting my own adapter on cloudflare, instead of forking the plugin.

[madox2]

There are tools like litellm that expose openai-compatible proxy to most of the llm providers. But like you said, you need to host it yourself or run locally