Users adding more OpenAIApi-compatible models to providers.py, "modding settings" from self-written external packages for example.

[aecorn]

trafficstars

Problem

Getting new models, that we provide from our own OpenAi-compatible API into the settings-gui.

Proposed Solution

Possibility of hooking into jupyter_ai_magics/providers.py, adding to the models-list somehow (our own add-on package for example). Before the GUI is rendered. https://github.com/jupyterlab/jupyter-ai/blob/8f3bfe075e2ca468f6ccd478c7255c52b7e3dbf9/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py#L453 Still passing in a different URL (our own) to the ENV VAR OPENAI_API_BASE.

Alternatively, make the settings-GUI typeable so we can add the models directly into the GUI. (Would force each of our users to manually type in the correct models...)

Additional context

We are trying to deploy opensource models using the FastChat-framework to our internal kubernetes clusters, when these are running, we would like to provide the jupyter-ai-extension with a different URL and the models of our choosing. FastChat provides an OpenAI-compatible API, which we would like to provide for our JupyterHub users on the same clusters. We have been able to reliably deploy the models of our choosing, with wildly different naming. The reason for us wanting to host our own models and API is because of the sensitive nature of our code and data. The main use will be generating and "transpiling" into Python code, from existing code bases in SAS.