Support for any OpenAI compatible endpoint

[Awwtifishal]

All supported providers ultimately use the same API (OpenAI compatible), so why not just support one single API with a choice for the provider endpoint (base URL)?

Also, I would like to choose the endpoint for LLM and for embeddings separately, to be able to use llama.cpp for example (which can only load one model).

[sairin1202]

Good point! We will put it into our roadmap and publish memU implemented llm endpoint soon! Will notify when we finished. Thank you for your feedback

[Awwtifishal]

Looking at this commit I'm not sure if you understood my message. All we need is just that, but for embedding instead of the LLM. Just an env variable for the endpoint (base URL) for the OpenAI embedding API which is different than OPENAI_BASE_URL. For example something like OPENAI_EMBEDDING_BASE_URL, which falls back to OPENAI_BASE_URL for the people that don't need a different endpoint.

[Awwtifishal]

I managed to make a proxy that can serve both the LLM and embeddings from the same endpoint (openai compatible), and it seems that the server tests work.

I'm not closing the issue because of other folks that want to try a local deployment.