~KubernetesMockServer~ Kube API does not implement garbage collection / cascading of deletion based on owner references

[adriansuarez]

Is your enhancement related to a problem? Please describe

I have an application that uses owner references to manage groups of related resources. The main resource is set as an owner reference on the ancillary resources, and only the main resource is deleted, which we expect to propagate to the dependent resources.

This works as expected when using a real Kubernetes cluster, but I am unable to exercise this behavior in unit tests using the KubernetesMockServer implementation in io.fabric8:kubernetes-server-mock, because it does not seem to implement garbage collection / cascading of deletion based on owner references.

Describe the solution you'd like

If a resource is created that has metadata.ownerReferences entries, then the deletion of the referenced owner resources should cause the entry to be removed from owner references. When the final reference is removed, the dependent resource should be deleted.

Describe alternatives you've considered

I was able to bolt on the desired behavior by subclassing KubernetesCrudDispatcher and handling owner references inside of Dispatcher.dispatch(RecordedRequest). This works for me because I do not have to wait for a new release of the Fabric8 client before testing my application, but I am afraid of this being brittle and breaking when upgrading io.fabric8:kubernetes-server-mock.

It would be good to have a proper implementation of this inside of KubernetesCrudDispatcher. I glanced at the code and noticed that there is already some support for finalizers.

Additional context

For details on how garbage collection / cascading of deletion based on owner references is supposed to work, see the Kubernetes documentation:

• https://kubernetes.io/docs/concepts/architecture/garbage-collection/#owners-dependents
• https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/#owner-references

[shawkins]

Relates to #3266

[shawkins]

@adriansuarez ideally we'd like to not make much more of an investment in the crud mock server, and instead have users rely upon minikube, or other small k8 distribution, or jenvtest when full api-server or greater support is needed.

[adriansuarez]

@shawkins, thanks for the quick response.

We do use Minikube for coarse-grained end-to-end testing, but would like to avoid having a dependency on it for the unit tests, which should be self-contained and fast.

jenvtest seems like a promising alternative. I'll look into it.

[adriansuarez]

@shawkins, I took a look at jenvtest, and it seems to be wrapper for envtest which provisions etcd and the Kubernetes API server without any controllers like the one that implements garbage collection (see this note in kubebuilder docs). So I don't think switching to jenvtest would solve my problem, though I could add my own mock controller that implements garbage collection using the Kubernetes API.

I was considering pushing my change into the KubernetesCrudDispatcher and creating a PR, but maybe the right approach is to rewrite it as a controller, which would be usable with KubernetesMockServer and envtest.

[shawkins]

So I don't think switching to jenvtest would solve my problem

That's unfortunate.

but maybe the right approach is to rewrite it as a controller, which would be usable with KubernetesMockServer and envtest.

The drawbacks with doing this completely generically as a controller - the mock server has a very incomplete metadata facility, and it will require a separate watch per kind / apiVersion. Perhaps providing a list of things to manage would be simple enough compromise.

[adriansuarez]

To close the loop on this, we ended up migrating over to jenvtest and were able to implement a mock garbage collection controller that we combine with jenvtest to test the behaviors in our application that rely on Kubernetes garbage collection.

Thanks for your suggestion to use jenvtest and feel free to close this issue.

[manusa]

To close the loop on this, we ended up migrating over to jenvtest and were able to implement a mock garbage collection controller that we combine with jenvtest to test the behaviors in our application that rely on Kubernetes garbage collection.

Awesome, is this controller publicly available?

[csviri]

@adriansuarez an other way would be to run also the built in garbage collector controller for kubernetes in Jenvtest, my plan is to eventually try that.

[adriansuarez]

Awesome, is this controller publicly available?

It isn't currently, but we could upstream the code if there is interest. Note that the mock garbage collector is very simple and only supports our use case, which is background propagation of deletion. The other modes for cascading of deletion, orphan and foreground propagation, are not implemented.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!