Add requestOptions to OpenAI provider HTTP calls

[64q]

Validations

• [x] I believe this is a way to improve. I'll try to join the Continue Discord for questions
• [x] I'm not able to find an open issue that requests the same enhancement

Problem

Hi guys!

Thanks for your awesome tool! I'm currently using a custom vLLM endpoint with basic auth in front of it and I'm using the OpenAI provider.

It seems the requestOptions configuration is not injected in this provider (in fact, it's only used in the LlamaCpp provider). This prevents from injecting custom headers inside each request performed against the LLM endpoint in case you are using the OpenAI provider.

Solution

Add the requestOptions in the OpenAI provider fetches (or maybe make it even more generic in every provider?)

[sestinj]

@64q we do in fact use requestOptions for all LLM providers by injecting the same fetch to all of them (code here ).

I'm wondering if you're seeing this problem because you're trying to add an Authorization header? This will be overridden by the apiKey property, which translates into "Authorization": "Bearer xxx". I believe LlamaCpp may just be the only one which doesn't use the apiKey property, which is why it seemed to work. More information here: https://continue.dev/docs/model-setup/configuration#authentication

[sestinj]

this does bring up a good point though that we probably ought to have the custom headers override apiKey, rather than the other way around!

[64q]

@64q we do in fact use requestOptions for all LLM providers by injecting the same fetch to all of them (code here ).

I'm wondering if you're seeing this problem because you're trying to add an Authorization header? This will be overridden by the apiKey property, which translates into "Authorization": "Bearer xxx". I believe LlamaCpp may just be the only one which doesn't use the apiKey property, which is why it seemed to work. More information here: https://continue.dev/docs/model-setup/configuration#authentication

You are completely right, I'm adding an Authorization header and the OpenAI provider seems to override the header with the Bearer one, even if I've a custom Authorization header already declared in my config file.

Thank you for your quick answer!

[64q]

this does bring up a good point though that we probably ought to have the custom headers override apiKey, rather than the other way around!

If you have any idea whether you will address this issue, please do not hesitate to let me know. 🤗

[sestinj]

@64q This has now been addressed in a recent commit! https://github.com/continuedev/continue/commit/e150f7d26fb076bcdfb1832013ecbff2b93b060d