kksasa

如何保持模型在内存中或立即卸载？ 默认情况下，模型在内存中保留5分钟后会被卸载。这样做可以在您频繁请求LLM时获得更快的响应时间。但是，您可能希望在5分钟结束之前释放内存或无限期保持模型加载。使用/api/generate和/api/chat API端点的keep_alive参数来控制模型在内存中保留的时间。 keep_alive参数可以设置为： 一个持续时间字符串（例如"10m"或"24h"） 一个以秒为单位的数字（例如3600） 任何负数，将会无限期保持模型在内存中（例如-1或"-1m"） '0'，将在生成响应后立即卸载模型 例如，要预加载模型并保留在内存中，请使用 curl -d '{"model": "llama3", "keep_alive": -1}' 要卸载模型并释放内存，请使用： curl -d '{"model": "llama3", "keep_alive": 0}' 或者，您可以通过在启动Ollama服务器时设置 OLLAMA_KEEP_ALIVE 环境变量来更改所有模型加载到内存中的时间。OLLAMA_KEEP_ALIVE 变量采用与上述keep_alive参数相同的参数类型。请参考上述说明如何配置Ollama服务器以正确设置环境变量。 如果您想覆盖 OLLAMA_KEEP_ALIVE 设置，可以在/api/generate或/api/chat API端点使用...

faced same issue too

if someone can run ollama for graphrag. Please share the steps to us :）

Confused me as well. How does graphrag for db management. such as add/insert/delete

marked it for the answer

> Gradio itself (which Kotaemon is built upon) provide automated API wrapper. See https://www.gradio.app/guides/getting-started-with-the-python-client. > > Here is a quick code snippet that you can try. (with Kotaemon running in...

> > Gradio itself (which Kotaemon is built upon) provide automated API wrapper. See https://www.gradio.app/guides/getting-started-with-the-python-client. > > Here is a quick code snippet that you can try. (with Kotaemon running...

no , it is not a real stream, just print all output from list..

> To use streaming mode, https://www.gradio.app/guides/getting-started-with-the-python-client#generator-endpoints may be this can help. We have not test this API in detail so you would have to perform experiments yourself. Thanks for the...

I got the same, but I downloaded user installer app (https://github.com/Cinnamon/kotaemon/releases) to run sctipt to install ENV then in vs code I changed to env = kotaemon-app\install_dir\env