AI Provider Selection

[Josh-XT]

Duplicates

• [X] I have searched the existing issues

Summary 💡

It would be great if there was a way to have pluggable AI Providers to allow people to add other ones such as llama, oobabooga, or others to run models their prompts over API . With this, having different "prompt files" per model since some models respond to prompts differently than others is also helpful.

Additionally, pluggable embeddings to give people the option to run locally or use different embedding models.

I tried to make it easy to work with in my BabyAGI port. If any of the code in my examples is useful for this, absolutely feel free to use it.

Examples 🌈

I started to do this with BabyAGI. Trying to keep up with all of the large changes to the code here is making it impossible for me to add this after each merge.

https://github.com/Josh-XT/Agent-LLM (Note - under development)

Example of openai provider https://github.com/Josh-XT/Agent-LLM/tree/main/provider/openai

Example of oobabooga text generation web UI provider https://github.com/Josh-XT/Agent-LLM/tree/main/provider/oobabooga

Example of openai embedding https://github.com/Josh-XT/Agent-LLM/tree/main/embedding/openai

Example of longformer embedding to run locally https://github.com/Josh-XT/Agent-LLM/tree/main/embedding/longformer

Motivation 🔦

Giving people the ability to choose a different AI provider will enable people to use Auto-GPT on their own custom models or other models out in the wild besides GPT-3.5 and GPT-4. While GPT-4 is the best out there, it would help a lot for development to see how other models do given the same objectives for research and development.

[keldenl]

+1 on this, would love this to be added. I got a llama.cpp openai api drop-in replacement written at https://github.com/keldenl/gpt-llama.cpp but embeddings is something i'm stuck on. it works off and on and i think it's likely related to the dimensions :/

[Josh-XT]

+1 on this, would love this to be added. I got a llama.cpp openai api drop-in replacement written at https://github.com/keldenl/gpt-llama.cpp but embeddings is something i'm stuck on. it works off and on and i think it's likely related to the dimensions :/

I started my own project with this idea in mind and have added quite a bit to it. https://github.com/Josh-XT/Agent-LLM

[Boostrix]

related #2158, #2621, #2777 etc

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions[bot]]

This issue was closed automatically because it has been stale for 10 days with no activity.