Request for documentation/tips on how to renew the agent SSL certificate

[pipern]

I've read through https://docs.stackable.tech/agent/configuration.html and do not yet understand enough to see how to renew the agent certificate which has expired:

# openssl x509 -enddate -noout -in /etc/stackable/stackable-agent/secret/agent.crt 
notAfter=Sep  7 11:02:35 2021 GMT

I think it was created around here:

Sep 06 15:45:14 vm1 stackable-agent[7509]: [2021-09-06T15:45:14Z INFO  stackable_agent] Successfully bootstrapped TLS certificate: TLS certificate requires manual approval. Run kubectl certificate approve>
Sep 06 15:45:14 vm1 stackable-agent[7509]: [2021-09-06T15:45:14Z INFO  kubelet::bootstrapping] Got modified event, but CSR for serving certs is not currently approved, 4.777867ms remaining
Sep 06 15:45:35 vm1 systemd[1]: Started snap.kubectl.kubectl.3b03a490-5f60-4849-8f01-69b4d45de525.scope.
Sep 06 15:45:35 vm1 stackable-agent[7509]: [2021-09-06T15:45:35Z INFO  kubelet::bootstrapping] Got modified event, but CSR for serving certs is not currently approved, 21.366446172s remaining
Sep 06 15:45:35 vm1 systemd[1]: snap.kubectl.kubectl.3b03a490-5f60-4849-8f01-69b4d45de525.scope: Succeeded.
Sep 06 15:45:35 vm1 stackable-agent[7509]: [2021-09-06T15:45:35Z INFO  stackable_agent] Successfully bootstrapped TLS certificate: received TLS certificate approval: continuing
Sep 06 15:45:35 vm1 stackable-agent[7509]: [2021-09-06T15:45:35Z INFO  kubelet::node] Successfully created node 'vm1'

so maybe something should be renewing it frequently?

I noticed this when trying:

# kubectl logs -n openshift-azure-logging po/mdsd-ng5dx fluentbit
Error from server: Get "https://10.0.8.4:3000/containerLogs/openshift-azure-logging/mdsd-ng5dx/fluentbit": x509: certificate has expired or is not yet valid: current time 2021-09-08T09:45:39Z is after 2021-09-07T11:02:35Z

(mdsd-ng5dx is a pod that OpenShift puts on. If that can work or not, since we're running stackable-agent and not the openshift kubelet, I think is an unrelated question to the certificate expiry)

[soenkeliebau]

Hi Nick, currently this is not auto renewed, an issue for that exists upstream though: https://github.com/krustlet/krustlet/issues/553

You are totally right though, we should add documentation around how to perform a renew here as a first step.

Basically the steps should be:

• delete local certificate the agent uses
• delete certificatesigningrequest in kube-apiserver (if it still exists)
• restart agent
• sign csr's in kubernetes unless autosigning is set up

[pipern]

Thanks, I'll try this out and then make a PR for some brief docs

[pipern]

Made https://github.com/stackabletech/agent/pull/295