Internal TLS expiration behaviour

[BRONSOLO]

When an internal TLS certificate expires and gets rotated by cert-manager, the operator does not seem to handle reloading the processes (e.g., /usr/bin/registry_DO_NOT_USE_GC serve) that reference these TLS certificates / mounted secrets. As a result, the harbor services fail to reach one another (with a 500-level TLS handshake error):

2022/11/17 13:46:29 http: TLS handshake error from 127.0.0.6:52783: remote error: tls: bad certificate

Once the certificate expires, the new certificate gets mounted into the pod as expected but the old certificate is still being served.

I'm curious what the expectation is here. Should all harbor cluster pods be rotated manually to pick up the new TLS certificates?

Thanks!

[thcdrt]

Hello,

It seems relative to issue https://github.com/goharbor/harbor-operator/issues/712

Thomas

[BRONSOLO]

Thanks @thcdrt! I had a look at that ticket as well as I also thought it could be related. I think #712 captures a separate failure state actually as I'm seeing that the non-CA certificates do get updated in the pod + mounted secrets as expected.

[tomkukral]

Just for a record when somebody find this issue ... It can workarounded by deploying stakater/reloader and restarting harbor's pods when secret (tls cert) changes.