Be able to decide when certificates will be rotated to minimize maintenance operations

[mtparet]

Proposal

When using multiple remotes ES clusters in different k8s clusters, we need to manually synchronize certificates for the remote configuration as described here https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-remote-clusters.html#k8s-remote-clusters-connect-external. But as ES clusters are not created all at the same time it means each certificate will be rotated at a different time and so it requires for each a maintenance operation at different times.

If we could indicate when the certificates are rotated this will enable having only one maintenance operation at one time to synchronise all certificates of all ES cluster being rotated at the same time.

Perhaps, we could add a ca-cert-rotate-at option https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-operator-config.html

[mtparet]

Related to https://github.com/elastic/cloud-on-k8s/issues/4675 but a little different because we keep the automation of rotating certificates.

[barkbay]

It's possible to use a custom CA to setup transport certificates. By using your own PKI, could it be a way to have a better control on the CA certificate lifecycle ?

[mtparet]

Indeed it could but it means we have to deal generate these ourselves which we wish to avoid. For now we will use the trick of deleting the secrets to force the operator to regenerate the certificate.