luffy

Environment="OLLAMA_NUM_PARALLEL=5" Environment="OLLAMA_MAX_LOADED_MODELS=2" Three people simultaneously used and loaded llama 3, which was very fast. The fourth person used to load codememma, which was very slow. localhost.localdomain Mon May 6 19:12:05...

It is meaningful when OLLAMA_NUM_PARALLEL =N is set. Multiple cards are more suitable than single cards to meet the configuration of OLLAMA_NUM_PARALLEL =100. 在 2024年5月9日 05:35，Daniel ***@***.***> 写道： Before v0.1.32,...

@44670 今天看到 1M出来了，哈哈