ragna
ragna copied to clipboard
Add OpenAI API compatible assistant
PoC - do not merge.
Details
- Download https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile/blob/main/mistral-7b-instruct-v0.2.Q5_K_M.llamafile (or host your own model locally)
-
export BASE_URL=http://localhost:8080
(or wherever your model is available locally) - Add
assistants = [ "ragna.assistants.OpenAIApiCompatible", ]
toragna.toml