Open LLama Compatibility

[markbastian]

Ollama has announced OpenAI API compatibility and the Llama API website has documentation examples using the Python OpenAI client. It would be great to see this library work with the Llama API as well.

[wkok]

Hello!

We have the ability to override the openai endpoint in options: https://cljdoc.org/d/net.clojars.wkok/openai-clojure/0.15.0/doc/usage-openai#options

Maybe it already works?

[jkreed]

I can confirm it works. Here is a snippet I was able to successfully run.

(api/create-chat-completion {:model "llama3.1"
                                            :messages [{:role "user"
                                                        :content "why are my eyes blue?"}]
                                            :stream true
                                            :on-next #(prn %)}
                                           {:api-key "key"
                                            :api-endpoint "http://localhost:11434/v1"})

[wkok]

I can confirm it works. Here is a snippet I was able to successfully run.

Thanks for confirming!