ovg-project
[TODO] Ollama integeration
It would be great to support Ollama with kvcached for local deployment of multiple LLMs.
Hi @jiarong0907, I would like to take on the Ollama integration issue, and I think it’ll be a great way for me to get more familiar with the codebase.
Hi @jiarong0907, I would like to take on the Ollama integration issue, and I think it’ll be a great way for me to get more familiar with the codebase.
Sounds good! Thank you!
A comment on Reddit mentioned that Ollama and LM Studio are actually based on llama.cpp with minor modifications. So maybe we can look into the delta there and hopefully the Ollama integration can be easily ported to Llama.cpp and LM studio.
https://www.reddit.com/r/LocalLLaMA/comments/1odddyg/comment/nkuocxy/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
