text-generation-webui
text-generation-webui copied to clipboard
Customizable/disablable server startup timeout (llama.cpp)
Description I tried to load a Q8 of DeepSeek-V3-0324 from my relatively slow storage array -- it returned the following error message after 8 hours.
Error loading the model with llama.cpp: Server health check timed out after 28800 seconds
https://github.com/oobabooga/text-generation-webui/blob/1aa76b3beb6f11a88b5a7b8df2b06a715e0dc123/modules/llama_cpp_server.py#L317C10-L317C38 With the new llama cpp loader system, attempting to load a model with an operation that takes over 8 hours will result in a timeout. Please add some method for disabling or increasing this timeout.