postgres-operator icon indicating copy to clipboard operation
postgres-operator copied to clipboard

Spilo pods don't define liveness, readiness probes

Open multi-io opened this issue 6 years ago • 5 comments

...which leads to situations where e.g. if Patroni incessantly tries and fails to start postgres, this won't be visible in the pod's status (and hence in generic monitoring setups that don't specifically track e.g. the DB's replica count).

At the minimum I'd think the liveness probe should track the supervisord status (pod is live iff all services are running) and the readiness probe should track the Patroni state of the member (pod is ready iff the state is "running").

multi-io avatar Jun 13 '19 16:06 multi-io

The absence of readiness probe impairs the PodDisruptionBudget logic, btw.

So I've defined those probes as follows. Maybe it's somewhat an overkill.

         livenessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - |-
              /usr/local/bin/supervisorctl status &&
              /usr/bin/pg_isready &&
              /usr/bin/curl -s localhost:8008 | /bin/grep '"state": "running"'
          failureThreshold: 8
          initialDelaySeconds: 60
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 5

        readinessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - |-
              /usr/local/bin/supervisorctl status &&
              /usr/bin/pg_isready &&
              /usr/bin/curl -s localhost:8008 | /bin/grep '"state": "running"'
          failureThreshold: 3
          initialDelaySeconds: 45
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5

Crazy-Hopper avatar Nov 17 '19 03:11 Crazy-Hopper

https://github.com/zalando/patroni/issues/1159

baddlan avatar May 13 '20 01:05 baddlan

@neelasha-09 let's continue here.

As support for probes has been merged for Patroni they will likely land somewhen later this in the operator, too.

FxKu avatar Jul 16 '20 12:07 FxKu

@FxKu Hi, would you mind looking at this issue? It would be pretty helpful if we could add probe logics to Pods.

Thanks in advance.

chitoku-k avatar Apr 22 '21 17:04 chitoku-k

Just thought I'd mention here that the operator does add readiness probes now (added here). Liveliness probes aren't added though.

davidspek avatar Jul 15 '22 11:07 davidspek