ragna icon indicating copy to clipboard operation
ragna copied to clipboard

Add OpenAI API compatible assistant

Open nenb opened this issue 9 months ago • 0 comments

PoC - do not merge.

Details

  1. Download https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile/blob/main/mistral-7b-instruct-v0.2.Q5_K_M.llamafile (or host your own model locally)
  2. export BASE_URL=http://localhost:8080 (or wherever your model is available locally)
  3. Add assistants = [ "ragna.assistants.OpenAIApiCompatible", ] to ragna.toml

nenb avatar May 24 '24 19:05 nenb