Enhancement: Option to change OpenAI API Endpoint

[hahayusuf]

You can now set a custom API base URL through the env file using the API_BASE_URL variable. When kept blank (""), it will use the default base URL as per the previous implementation.

The README, and setup scripts were updated to incorporate the new variable.

[vercel[bot]]

@hahayusuf is attempting to deploy a commit to the reworkd Team on Vercel.

A member of the Team first needs to authorize it.

[pengmaochang]

dude, this commit is very valuable for the countries that cannot connect to openai api url

[pengmaochang]

@hahayusuf the file "setup.sh" write like this may be better

#!/bin/bash
cd "$(dirname "$0")" || exit

echo -n "Enter your OpenAI Key (eg: sk...): "
read OPENAI_API_KEY
echo -n "Enter your API_BASE_URL IS YOU NEEDED ( press enter for default): "
read API_BASE_URL

NEXTAUTH_SECRET=$(openssl rand -base64 32)

ENV="NODE_ENV=development\n\
NEXTAUTH_SECRET=$NEXTAUTH_SECRET\n\
NEXTAUTH_URL=http://localhost:3000\n\
OPENAI_API_KEY=$OPENAI_API_KEY\n\
DATABASE_URL=file:../db/db.sqlite\n\
API_BASE_URL=$API_BASE_URL\n"
...

[Mohamedhoussein]

I am interested

[jondwillis]

This is also useful for analytics and caching layers such as https://helicone.ai

[hahayusuf]

Hi @hahayusuf, thanks for making this PR :) this is a very useful change

I have included some feedback. Also, there will likely be merge conflicts since a lot has changed since this PR was made

Thanks for your feedback!

Will let you know when it's ready to merge.

[hahayusuf]

@Jshen123 @pengmaochang

The merge conflicts have now been resolved and can be merged automatically.

I have refactored this to use the ModelSettings interface for setting the customBaseUrl.

• Input field created for customBaseUrl under advanced settings in settings dialog.
• Fixed other comments related to "Copy!" message when clicking the copy button.

[hahayusuf]

@Jshen123

Thanks for your feedback.

I have added your "minor tweaks" commit.

Then I noticed some merge conflicts, which I have now resolved.

[awtkns]

I will take a look at this shortly!

[hahayusuf]

I have a slight security concern about allowing users to change the base url in our production environments. Doing so could potentially expose our keys if we allow them to change the baseurl + not have to input their own key. Ideally we will probably want to keep this client side if possible. Thoughts @asim-shrestha?

It is a valid concern @awtkns. We can protect against it by implementing two safeguards: -Frontend: hiding the custom url input unless a customApiKey is set. -Backend: when settings.customBaseUrl is set, env.OPENAI_API_KEY cannot be used, only customApiKey. A simple guard clause should do the trick.

[Jshen123]

I have a slight security concern about allowing users to change the base url in our production environments. Doing so could potentially expose our keys if we allow them to change the baseurl + not have to input their own key. Ideally we will probably want to keep this client side if possible. Thoughts @asim-shrestha?

It is a valid concern @awtkns. We can protect against it by implementing two safeguards: -Frontend: hiding the custom url input unless a customApiKey is set. -Backend: when settings.customBaseUrl is set, env.OPENAI_API_KEY cannot be used, only customApiKey. A simple guard clause should do the trick.

@awtkns that's a really good callout. @hahayusuf agree with your suggested solution (only change here would be to use either disable or disable + hide)

@awtkns @asim-shrestha, if we are not comfortable with custom URL in production, perhaps we can rollback URL input feature and release the rest of the code (so users can at least use their own URLs locally). Thoughts?

[hahayusuf]

Update: I have implemented both safeguards discussed.

Thanks for your feedback @Jshen123

Look forward to hearing what others think.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agent-gpt	❌ Failed (Inspect)			Apr 24, 2023 2:41am

[asim-shrestha]

Hey sorry @hahayusuf, this has flown under the radar. Unfortunate because it seems we have a bunch more conflicts. I can have a look into manually fixing this stuff if you're not able.

Configuring endpoint locally via ENV seems like a good win. For endpoint in settings, given the safe guards it should be ok but maybe some unit tests are in order to validate this. Otherwise, perhaps we can provide a dropdown of options people can use. Not sure what other endpoints they might try.

[hahayusuf]

Hey @asim-shrestha

Thanks for offering to resolve some of the conflicts.

I agree, using env to set a custom base url should be a good enough win.

Will work on that. May need help ironing out conflicts.

[asim-shrestha]

Awesome!

[awtkns]

Closed as this has been already implemented