text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Customizable/disablable server startup timeout (llama.cpp)

Open computer-whisperer opened this issue 8 months ago • 0 comments

Description I tried to load a Q8 of DeepSeek-V3-0324 from my relatively slow storage array -- it returned the following error message after 8 hours.

Error loading the model with llama.cpp: Server health check timed out after 28800 seconds

https://github.com/oobabooga/text-generation-webui/blob/1aa76b3beb6f11a88b5a7b8df2b06a715e0dc123/modules/llama_cpp_server.py#L317C10-L317C38 With the new llama cpp loader system, attempting to load a model with an operation that takes over 8 hours will result in a timeout. Please add some method for disabling or increasing this timeout.

computer-whisperer avatar Apr 26 '25 15:04 computer-whisperer