operator-lifecycle-manager icon indicating copy to clipboard operation
operator-lifecycle-manager copied to clipboard

Fix finalizer processing

Open tmshort opened this issue 1 year ago • 2 comments

Description of the change:

Handle CSV deletion during sync.

Motivation for the change:

There are two reconcilers for the CSV, one is controller-runtime based, the other is based on queueinformer. A finalizer was added to the CSV and it's handled by the controller-runtime reconciler.

However, the queueinformer-based reconciler may take a while to do its processing such that the CSV may be deleted and the finalizer run via the controller-runtime reconciler. (This still takes <1s.)

This causes problems when CSV being synced is deleted. The queueinformer attempts to sync RBAC to the stale CSV. The proper (BIG/risky) fix is to consolidate the two reconcilers. A less risky fix is to query the lister cache to see if the CSV has been deleted (or has a DeletionTimestamp), and not do the RBAC updates if that's the case.

Architectural changes:

None.

Testing remarks:

Reviewer Checklist

  • [ ] Implementation matches the proposed design, or proposal is updated to match implementation
  • [ ] Sufficient unit test coverage
  • [ ] Sufficient end-to-end test coverage
  • [ ] Bug fixes are accompanied by regression test(s)
  • [ ] e2e tests and flake fixes are accompanied evidence of flake testing, e.g. executing the test 100(0) times
  • [ ] tech debt/todo is accompanied by issue link(s) in comments in the surrounding code
  • [ ] Tests are comprehensible, e.g. Ginkgo DSL is being used appropriately
  • [ ] Docs updated or added to /doc
  • [ ] Commit messages sensible and descriptive
  • [ ] Tests marked as [FLAKE] are truly flaky and have an issue
  • [ ] Code is properly formatted

tmshort avatar Feb 22 '24 15:02 tmshort

/hold

tmshort avatar Feb 22 '24 16:02 tmshort

This is an improvement, but it's not perfect... there's still a race condition.

tmshort avatar Feb 22 '24 17:02 tmshort

/unhold

tmshort avatar Feb 26 '24 21:02 tmshort

ping @kevinrizza ?

tmshort avatar Feb 28 '24 18:02 tmshort

/approve /lgtm

kevinrizza avatar Feb 28 '24 18:02 kevinrizza