Enhancement: Custom api endpoints

[FetchFast]

Suggestion: Enhancement

An example of a custom endpoint would be: LLaMA 13B https://api.runpod.ai/v2/yourServer/runsync

It would:

1. allow Intellibar to be used in places where the OpenAI API is not available.
2. cheaper than openai, possibly by a lot

[astoilkov]

We've been thinking about this idea ourselves.

I'll be glad if more people join the discussion and share their use cases. That will help us with the details, direction, and importance.

[FetchFast]

Sure. I'd like to

1. point the endpoint at my serverless runpod llama, because it's vastly cheaper than openAI
2. point the endpoint to my server running memgpt, loaded with my own reference material to give me an unlimited context window.

Does that sound useful?

[astoilkov]

Yes, thanks! I hope others join the discussion as well.

[rb81]

+1 Ollama and more external endpoints.

[bikevit2008]

i need openai compatible endpoint to connect my self-hosted models