Revisit decision to use finalizers to manage bundle storage

[joelanford]

As an outcome of #292, we began using finalizers to manage the underlying storage of bundles. With the changes in that PR, we can no longer rely on kubernetes garbage collection to clean up all bundle contents from the cluster because the new LocalDirectory storage implementation is essentially in off-cluster location that only the provisioner is aware of.

We brainstormed the idea of clearing bundle storage via a DELETE event handler, and backing that up with a periodic reaping mechanism to catch events that we missed. But after experimenting with that idea, it became clear that we'd have to expand the definition of the storage interface to handle listing all of the currently stored bundles, which is something we weren't necessarily prepared to do.

In this issue, let's discuss the whether we think finalizers are a reasonable solution to avoid "zombie" bundles sticking around in storage and also what reasonable alternatives would be. What are the pros and cons of the various choices?

There's further context for this issue at https://github.com/operator-framework/rukpak/pull/292#issuecomment-1116137822.

[github-actions[bot]]

This issue has become stale because it has been open 60 days with no activity. The maintainers of this repo will remove this label during issue triage. Adding the lifecycle/frozen label will cause this issue to ignore lifecycle events.

[tylerslaton]

So the context captured in this PR should allow for this work to be actionable and help us to come to a decision. If there are any needed follow-up questions lets go ahead and direct them at @operator-framework/rukpak-maintainers.

[github-actions[bot]]

This issue has become stale because it has been open 60 days with no activity. The maintainers of this repo will remove this label during issue triage or it will be removed automatically after an update. Adding the lifecycle/frozen label will cause this issue to ignore lifecycle events.