on.delete handlers without finalizers?

[OmegaVVeapon]

Question

What problem do you currently face and see no solution for it? So, without going too deep into the details (although I can if you want me to), due to "outside factors" it would be helpful to have the deletion handlers run without relying on the contract that finalizers offer.

If possible, explain what other ways did you try to solve the problem? From reading the docs, I noticed the optional=True argument that could be passed to the on.delete handler to avoid adding finalizers to the resources but my understanding was that the handler would still be called, it just wouldn't be as "safe" as when the finalizers were enforced by k8s. Instead, I noticed that the delete handlers weren't called at all. Is that the expected behaviour?

Then I tried to simply use a generic event handler that only fires when we delete objects

def resource_is_deleted(event, **_):
    return event['type'] == 'DELETED'

@kopf.on.event('', 'v1', 'configmaps', when=resource_is_deleted)
def delete_fn(body, logger, **_):
    delete_file(body, body['kind'], logger)

and this seems to work just as well as the on.delete handler...

I'm sure it's not as safe since there's no blocking enforced by k8s on that delete_fn completing before the object is marked for deletion but it's the best I could come up with...

Is there a better/safer way I could go about this?

Using kopf version 1.29.2

Checklist

• [x] I have read the documentation and searched there for the problem
• [x] I have searched in the GitHub Issues for similar questions

Keywords

• finalizers

[nolar]

Hello. Sorry, I probably didn’t get what is the question or a supposed behavior alternative to the existing one.

Generally, you are right: with optional=True, there are no finalizers, and the execution of deletion handlers depends on luck — whatever is faster: the operator to trigger the deletion handlers or Kubernetes to wipe the resource.

In a clean custom resource case, Kubernetes usually wins.

The operator has more chances only if the resource is blocked by other finalizers of other controllers/operators, e.g. when pods terminate after being marked for deletion (it takes up to 30 seconds).

The deletion handler should still be called on a last “goodbye” event — at least one, at least once; but no retries or second chances. Is it not called?

[OmegaVVeapon]

Yeah, sorry, I could have definitely been clearer.

So, I tried the following things:

on.delete with finalizers (normal use case)

@kopf.on.delete('', 'v1', 'configmaps', when=kopf.all_([label_is_satisfied, resource_is_desired]))
@kopf.on.delete('', 'v1', 'secrets', when=kopf.all_([label_is_satisfied, resource_is_desired]))
def delete_fn(body, logger, **_):
    logger.info("A secret/resource was deleted!")

Result: I see the log appear when I delete the matching configmaps/secrets. ✅

on.delete without finalizers. (optional=True)

@kopf.on.delete('', 'v1', 'configmaps', optional=True, when=kopf.all_([label_is_satisfied, resource_is_desired]))
@kopf.on.delete('', 'v1', 'secrets', optional=True, when=kopf.all_([label_is_satisfied, resource_is_desired]))
def delete_fn(body, logger, **_):
    logger.info("A secret/resource was deleted!")

Result: The log never appears when I delete the matching resources ❓

on.event with a filter for DELETED events

@kopf.on.event('', 'v1', 'configmaps', when=kopf.all_([label_is_satisfied, resource_is_desired, resource_is_deleted]))
@kopf.on.event('', 'v1', 'secrets', when=kopf.all_([label_is_satisfied, resource_is_desired, resource_is_deleted]))
def delete_fn(body, logger, **_):
    logger.info("A secret/resource was deleted!")

Result: I see the log appear when I delete the matching configmaps/secrets. ✅

---

So yeah, I'm not seeing the handler get executed at all with optional=True.

It's no big deal though, the on.event approach is working perfectly so I think I'll just go for that. In any case, I suspect that users that are trying to do what I am (deleting resources without finalizers) are extremely rare.

So I'm good for now. Let me know if there's any additional info you'd like me to gather, otherwise, feel free to close this question.

[nolar]

Thanks for clarifying. Indeed, I could reproduce this locally with kubectl alone. I think I know what causes it.

When a resource is deleted with a finalizer, .metadata.deletionTimestamp is set. Kopf uses that field to detect the ongoing deletion.

When a resource is deleted without finalizers, it is just deleted, and that's it. Only a DELETED event is sent.

However, Kopf interprets the DELETED event as a "resource is gone" reason/cause type, i.e. not the actual deletion. And for this, it just logs that the resource is gone, without even trying to invoke any handlers.

This implementation goes deep down to the historic roots of the framework, where this was the last step of deletion (always with finalizers). So it is not something broken recently.

~But it can be fixed.~ Actually, while writing about how it can be fixed, I remembered that it was discussed earlier. The outcome was that it would break the semantics of handlers. When an event of type "DELETED" is received, the resource does not exist anymore, so it would be the handling of a ghost. But without this fix, the optional deletion handlers are useless, as you have properly noticed.

I will need to find that original discussion and refresh my memories and rethink how the deletion should work — a bit later, not right now (you have a working workaround, as I understand).