Hagen Hübel

I've conducted extensive testing today and encountered an issue with the Ollama model while running a FastAPI-based Python API on a GPU machine (RTX 4000 from Hetzner). Here are the...

Btw, I found also out, that the invokation of `ollama.chat` is blocking the full python process. I can not even call another endpoint during that time, not even the "/docs"-endpoint...

I was running into the same, reported here https://github.com/abetlen/llama-cpp-python/issues/1693

For the sake of completeness: if someone is looking for a llama.cpp binding that works with CUDA support, no matter which underlying programming language, I can recommend the NodeJS bindings:...

thx, @sweatybridge > The permission issues are due to old versions of postgres. In order to fix them, you need to do a `db dump` locally after restoring from backup....