Better Key+Model management/UX

[rbren]

What problem or use case are you trying to solve? Users often choose a model, without having specified the necessary key for that model.

Describe the UX of the solution you'd like

• If LLM_MODEL is set in the backend, default the UI to that model
  • otherwise, force the user to choose a model before starting
• Whenever the user chooses a model in the UI, prompt them for an API key (and optionally base URL)

Do you have thoughts on the technical implementation?

• Save model, API key, base url to localStorage
  • each model should have its own settings
  • e.g.

  localStorage.add('models', JSON.stringify({
    'gpt-4': {
      'api_key': 'sk-...',
      'api_base': 'http://localhost:4000',
    },
  })
  

**Describe alternatives you've considered**

**Additional context**

[Yashwanth-Chandrakumar]

@rbren a small doubt, show we save the model details only if the user interacts with the UI or even if he sets it up in the backend.

[rbren]

IMO we should not send the backend API key back to the frontend, ever.

Only details entered into the frontend should be saved in localstorage.

[Yashwanth-Chandrakumar]

Ok thank you.

[Yashwanth-Chandrakumar]

In the config if the LLM_MODEL is not set, defaultly gpt-4-0125-preview gets assigned do i need to change this behaviour.

[rbren]

Good point. I think we want to keep that default for running main.py directly, but maybe not for the server?

[rbren]

This is doing pretty well now. Still room for improvement, but I'm going to close this one.