New policy: block removal of CRDs as long as CRDs are still in use

[Martin-Weiss]

Is your feature request related to a problem?

When uninstalling / re-installing applications that deliver CRDs - the uninstall of these application automatically removes all instances as well which could be data-loss.

Example: install cert-manager, then create a certificate, then uninstall cert-manager -> here the data is lost as the certificate with its cert / key will be deleted as part of the CRD removal.

Solution you'd like

So an admission webhook should block the CRD removal as long as corresponding instances are still available.

I assume this policy should be configurable in a way that we can say "allowed for CRDs of type xyz (maybe with regex..)" and "disallowed or abc (maybe with regex as well).

Default should be to block all CRD removal as long as instances are still using a given CRD and then a customer can selectively add some to allow if needed.

Alternatives you've considered

When using fleet a new feature will be available to bock CRD removal as well - but this is fleet specific and will not block every other way..

See https://github.com/rancher/fleet/issues/1978 and https://helm.sh/docs/chart_best_practices/custom_resource_definitions/

Anything else?

No response

[viccuad]

As talked, I'm wondering about having only 1 with settings. Something like:

Policy that rejects a provided list of resources if other resources exist. They can be CRDs. The resources to reject go into the policy rules, and the policy settings accepts an optional list of resources that would cause rejection. Example:

spec:
  rules:
  - apiGroups: ["apiextensions.k8s.io"]
    apiVersion: ["v1"]
    resources: ["*"] # this may need a specific list and not a *
    operations: ["DELETE"]
  settings:
    reject_if_resources_present: # optional, if empty, always rejects
      - certificates.cert-manager.io
      - issuers.cert-manager.io

I haven't fully looked into how one adds CRDs and the rules needed. Maybe I'm making a sound mistake.

[Martin-Weiss]

Looks great - most important is "allow deletion only if no resources of the given type exist" (default) - and "allow deletion anyway for the ones if some exist of the ones in the list..".

One concern here from an operational view is a helm uninstall that would fail with deletion a few CRDs.. how would a customer cleanup this later?

[viccuad]

They would need to manually delete the rest of dangling CRDs and reinstall the system I suppose (since deleting some CRDs means that the associated CRs were mistakenly deleted too). The usecase sounds like a good showcase for this policy indeed.