SuperAGI
SuperAGI copied to clipboard
TransformerOptimus
Model Endpoint & Settings at Agent Level
Realizing baked-in local model support is on the road-map, I want to emphasize a structural feature request. At the moment, the whole project uses one API key, and for those of us who modified our code a little, one API Endpoint. See #243 This won't be practical once local LLM support is introduced.
Each agent having have its own flexible set of settings, including BASE_URL (endpoint), API Key, model name (not just GPT-3.5 or GPT-4), would be super useful. Ideally the agent settings would be agnostic to the specific type of LLM, allowing an arbitrary number of setup parameters, pulled an agent-settings file. This should also take prompt variation into account, as some use #'s, some use <| .. etc...
That way, for example, a coding agent could point at a coding-specific local LLM, another points at an API-trained LLM, another agent points at Palm API, one points at GPT4, one at 3.5 for tasks, agents might use separate API keys, etc...
An agent settings popup in the UI would allow easy updating of those settings, even within a docker build without having to change a .env file and rebuild.
It would enable FrugalGPT, agent specialization, and much more.
@kenfink appreciates your suggestions, soon planning to support multi-models.
We might keep them in the tool config itself which will be stored at the organization level. As of now, we have organizations and project concepts but there aren't any options to create and add project or organization in frontend.
We are getting a models marketplace in v0.0.10 where you would be able to run any model available in the marketplace or even add your own model and run it. Although we will not be supporting the deployment of models on superAGI in this version.
