phospho icon indicating copy to clipboard operation
phospho copied to clipboard

Select different LLM provider from env variables

Open plbiojout opened this issue 1 year ago • 0 comments

You should be able to change the LLM provider without changing the code, just by specifying the model, base_url and api_key. Should work with any openai client compatible server (Mistral, Ollama, vLLM, together AI...)

Right now, you need to change how the OpenAI client is initialized (in the pipeline, not documented yet).

plbiojout avatar Mar 13 '24 06:03 plbiojout