serve icon indicating copy to clipboard operation
serve copied to clipboard

Health check for all API endpoints

Open heatxg opened this issue 3 years ago • 1 comments

🚀 The feature

At present, there is a health check /ping only for the inference endpoint. Add health check for other endpoints.

Motivation, pitch

There is currently no way to implement load balancer health checks for the metrics API prior to routing traffic to it.

Alternatives

No response

Additional context

It appears that this was considered (https://github.com/pytorch/serve/issues/735) and some enhancements were added but this does not address simple API calls such as curl http://0.0.0.0:8082/ping

heatxg avatar Jul 18 '22 14:07 heatxg

Probably good feedback to roll into present or future requirements for #1492

msaroufim avatar Jul 18 '22 16:07 msaroufim

On a slightly related thing, I'm trying to create a docker image which runs a torchserve instance given a model_store. The issue I'm facing is that, from outside the docker image, I want to run some unit tests but I do not know whether the workers are already up. The /ping endpoint already returns healthy but the workers are not up for example if they are downloading some checkpoints from huggingface etc. Is there a way to achieve this?

ozancaglayan avatar Nov 01 '22 09:11 ozancaglayan