compliantkubernetes-apps icon indicating copy to clipboard operation
compliantkubernetes-apps copied to clipboard

No cleanup of CertificateRequests

Open salehsedghpour opened this issue 1 year ago • 2 comments

Current behaviour

In a production environment, we can see that the CertificateRequests stay there forever. In long-run it might increase the memory consumption in cert-manager, as it mainly checks for those requests.

Expected behaviour

I was expecting to see unused CertificateRequests are being deleted after a while.

Steps to reproduce

1- Set up a cert that renews in a very short period 2- Let it run for a while 3- Check for the old requests.

Or simply just checking old certificateRequests in an old production environment.

Compliant Kubernetes Apps Version

0.35

Kubernetes Version

No response

Additional context

The solution could be using the revisionHistoryLimit that can be configured using an annotation. Please see:

  • https://cert-manager.io/docs/usage/ingress/#:~:text=cert%2Dmanager.io/revision%2Dhistory%2Dlimit We may need updated documentation for application developers to know about this, or a gatekeeper policy that checks for this annotation.

salehsedghpour avatar Feb 21 '24 13:02 salehsedghpour

@elastisys/goto-ingress-auth

  • Is this a problem a problem for us at the moment ?
  • Do we want to get rid of old certificate requests ?

Pavan-Gunda avatar Dec 02 '24 13:12 Pavan-Gunda

I don't think this is a problem in most cases, Welkin apps only renew certificates every 60 days so you only end up with ~6 Certificaterequests per app that uses Certificates each year. But an application developer might have applications that renew Certificates every day or several times every day. Once you start to reach thousands of Certificaterequests (and Order resources as well I believe) you might start noticing that Cert-manager memory is higher than usual as it watches this resources and might get OOMkilled until you clean up some resources.

This can however be fixed in the individual Certificate resources by setting the linked annotation. So we could configure this on Welkin apps certificates to help reduce the risk for this, or have alerts or gatekeeper policies to avoid this issue. But I do not think this is something that needs high prio, but this might be good to be aware about. I also think a lot of memory issues with cert-manager has been improved since apps v0.35 (cert-manager v1.11.0), see this issue for example.

anders-elastisys avatar Dec 02 '24 16:12 anders-elastisys

Will be fixed by https://github.com/elastisys/compliantkubernetes-apps/issues/2812 as the default revision history limit has been updated: https://cert-manager.io/docs/releases/release-notes/release-notes-1.18/#the-default-value-of-certificatespecrevisionhistorylimit-is-now-1

Ajarmar avatar Oct 21 '25 13:10 Ajarmar