Make `api_key` and `api_base` configurable via `initialize` event

[rbren]

What problem or use case are you trying to solve? We should allow the frontend to send an API key alongside the model name

Describe the UX of the solution you'd like See: https://github.com/OpenDevin/OpenDevin/issues/208

Do you have thoughts on the technical implementation? Pull out api_key and api_base parameters here and pass it to the LLM: https://github.com/OpenDevin/OpenDevin/blob/02a0367757b665171e9dff28fe434713382f2b92/opendevin/server/session.py#L116

https://litellm.vercel.app/docs/set_keys#environment-variables

[Umpire2018]

I would like to give it a shot but i need more specific guidance.

1. What do you mean by pull out api_key and api_base parameters here and pass it to the LLM: ?

2. Right now we have api_key and api_base read in environment variables , maybe a better solution is to use a class to that reads configuration information from environment variables once and manages these configurations globally ?

[rbren]

Good questions!

Currently we have an LLM class which gets initialized with a model_name, api_key, and api_base. session.py creates a new LLM every time there's a new session, or it gets an initialize event.

The environment variables set the defaults for those values. But whoever is constructing the LLM instance can override them.

So you'd basically need to:

• in the frontend, send an initialize event with api_key and api_base
• in session.py, pass those values to the LLM constructor