operator-controller icon indicating copy to clipboard operation
operator-controller copied to clipboard

✨ WIP Improved error handling around installedBundle and reconcile modularization

Open bentito opened this issue 1 year ago • 4 comments

Description

WIP, because:

  • I'd like feedback on whether the reconciler() loop modularization seems good and whether the proposed logic flow changes match up with https://github.com/operator-framework/operator-controller/issues/1027 , which is restated below.

  • Unit tests are still broken. Just looking for feedback on overall approach.

  • This branch/PR cherry picks all the new commits (as of now) from branch/PR: https://github.com/operator-framework/operator-controller/pull/1007

  • vs. 1007 this PR also merged a slightly updated reconciler() from main around preflight implementation, so we should check logic still as intended


  1. Improved Error Handling
  • State Preservation: The installedBundle information is fetched early on and preserved throughout the reconciliation process. Even if errors occur later, the status accurately reflects the previously installed bundle.

  • Partial Progress Reporting: Each step has its own error handling and status update logic. If an error occurs in one step, the status is updated accordingly (e.g., setResolvedStatusConditionFailed, setInstalledStatusConditionFailed), but the reconciliation process continues where possible. This ensures that partial progress is reported.

  • Progressing Status: The setInstalledStatusConditionProgressing function is used to indicate when a new version is resolved but not yet installed. The clearInstalledStatusConditionProgressing function is called when the installation is complete or if the resolved and installed bundles match.

  1. Modularization
  • Helper Functions: The reconcile function is broken down into smaller, self-contained helper functions (handleFinalization, resolveBundle, validateAndCheckDeprecation, unpackBundle, runPreflightChecks, installOrUpgrade, and watchDynamicResources). Each function handles a specific task, making the code more organized, readable, and testable.
  1. Helm Secret as Source of Truth
  • GetInstalledBundle: The InstalledBundleGetter.GetInstalledBundle function is used to retrieve the installed bundle information directly from the Helm secret, ensuring that the status accurately reflects the actual installed bundle.
  1. Comprehensive Status Updates
  • Status Updates at Each Step: Throughout the reconcile function and its helpers, status conditions are updated to reflect the progress and outcome of each step. This provides detailed information about the reconciliation process, even if errors occur.

  • Progressing Status: The TypeProgressing condition is used to clearly indicate when an installation or upgrade is in progress.

Reviewer Checklist

  • [ ] API Go Documentation
  • [ ] Tests: Unit Tests (and E2E Tests, if appropriate)
  • [ ] Comprehensive Commit Messages
  • [ ] Links to related GitHub Issue(s)

bentito avatar Jul 10 '24 14:07 bentito

Deploy Preview for olmv1 ready!

Name Link
Latest commit 0469c53620925e8faf21a912dd499174cb7c319e
Latest deploy log https://app.netlify.com/sites/olmv1/deploys/668ea3a86f0d370008b11e66
Deploy Preview https://deploy-preview-1030--olmv1.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar Jul 10 '24 14:07 netlify[bot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-merge-robot avatar Jul 13 '24 22:07 openshift-merge-robot

@bentito are you going to rebase this?

tmshort avatar Jul 26 '24 14:07 tmshort

@bentito are you going to rebase this?

I'm waiting to see about Bryce's refactoring from last week

bentito avatar Jul 26 '24 17:07 bentito

closing this as stale - please re-open if we still want to get this down =D

perdasilva avatar May 22 '25 07:05 perdasilva