grpc-k8s-health-check icon indicating copy to clipboard operation
grpc-k8s-health-check copied to clipboard

Dont check dependent services

Open eccles opened this issue 4 years ago • 1 comments

I have just done an evaluation of kubernetes best practices and checking dependent services as done here appears to be incorrect.

Better practice seems to be to use an initContainer to make sure that preconditions are met. Also checking dependencies leads to cascading failures.

Ref: https://medium.com/@xcoulon/initializing-containers-in-order-with-kubernetes-18173b9cc222

and cascading failures: https://srcco.de/posts/kubernetes-liveness-probes-are-dangerous.html

eccles avatar Dec 02 '19 17:12 eccles

So this repo is primarily an example for how to do gRPC health checks and how to integrate them with Kubernetes. I feel like adding an initContainer to this might detract from the goal.

In regards to the readiness check validating a DB. While there is certainly more to it in critical applications like setting DB check failure thresholds, and using async checks like https://github.com/InVisionApp/go-health does. I don't think making blanket statements like it's wrong to do live/readiness checks that validate dependencies is correct. In fact I strongly disagree with that stand point.

I'll look at if it's worth making the DB check in this example async, but I kind of feel like that would over complicate the implementation and confuse first timers more than help them.

madflojo avatar Dec 04 '19 06:12 madflojo