Lightning-AI
shutdown LitServe programmatically
🚀 Feature
Multiple users have requested for a way to stop a running LitServe programatically rather terminating the process manually via keyboard interrupt such as https://github.com/Lightning-AI/LitServe/issues/491.
One way is to have an endpoint /shutdown which gracefully terminates and cleans up everything.
Motivation
Pitch
Alternatives
Additional context
Hey @aniketmaurya, I've done some research and implemented a shutdown endpoint that successfully terminates the processes. However, there are still some exceptions thrown during shutdown due to Uvicorn's inherited run command.
Specifically, when using os.kill(os.getpid(), signal.SIGINT) to stop the server, exceptions are expected, but they can't be suppressed because we don't have control over Uvicorn’s internal run behavior.
Is this acceptable as is, or would you prefer that we find a way to suppress those exception messages?
Hi @kumarrah2002 , About the exceptions during shutdown — I think it might be acceptable as long as we ensure that: No child processes or threads are left running and port is fully released for reuse.
btw, @aniketmaurya One question out of curiosity: Incase if this endpoint is accessible publicly, wouldn’t it pose a security risk by allowing anyone to shut down the server?
great point @bhimrazy! let's add additional authorization for /shutdown.
@aniketmaurya An idea I had for security was to generate a unique API key that is only for the shutdown endpoint (e.g. SHUTDOWN_API_KEY) that is saved to the user's environment or logged in the terminal. When the shutdown endpoint is called, this key must be present or there will be a 401 Unauthorized error. I discussed this with @bhimrazy but wanted to get your perspective.
CC: @Borda
sounds good @kumarrah2002! additionally, shutdown endpoint should be optional and only enable that if the user provides an argument such as LitServer(enable_shutdown_api=False).
Link to docs to shutdown LitServe with code - https://lightning.ai/docs/litserve/how-tos/shutdown-with-code