pxc container's `livenessProbe` can not be changed by PerconaXtraDBCluster's `livenessProbes`

[hulucc]

As the title.

[awlleon]

readinessProbes is also like this, but it works fine in version 1.11 of PXC.

[spron-in]

Hey folks. I was able to reproduce it.

I changed it in cr.yaml:

pxc:
...
    readinessProbes:
      timeoutSeconds: 16
      periodSeconds: 31

    livenessProbes:
      timeoutSeconds: 6
      periodSeconds: 11

I see it is properly set in the custom resource in k8s.

But it is not reflected in the container.

    livenessProbe:
      exec:
        command:
        - /var/lib/mysql/liveness-check.sh
      failureThreshold: 3
      initialDelaySeconds: 300
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 5

At the same time I see env variables are set:

   - name: LIVENESS_CHECK_TIMEOUT
      value: "6"
    - name: READINESS_CHECK_TIMEOUT
      value: "16"

@egegunes @hors any thoughts before I create a JIRA bug?

[hors]

@spron-in @hulucc it was fixed under https://perconadev.atlassian.net/browse/K8SPXC-1345 and will be available in the next release.

[spron-in]

Awesome! Thanks @hors !

[johnlaur]

Thank you @hors ! When we restore a foreign backup onto a pxc-operator managed cluster the inability to adjust the readiness probes gave a very slim window where we could connect into the pxc container of the pod and get the grants restored before the failing probe took out the container. I discovered this issue attempting to lengthen the timeout.

It would be beneficial if the operator could detect and possibly remediate the condition where the grants required by the operator are not present in the database; however I do not know if that is already possible by some existing mechanism I have not yet discovered. Is it possible to use a prestop hook on the restore job I wonder?