ImageUpdateAutomation enhancements with v1beta2 API

[darkowlzz]

Similar to other Flux controllers, image-automation-controller is undergoing refactoring. The refactor includes overall code organization changes and design enhancements. This issue discusses the enhancements and API changes.

Refactor PR https://github.com/fluxcd/image-automation-controller/pull/647.

Update ResultV2

In order to address https://github.com/fluxcd/image-automation-controller/issues/437 (inclusion of previous and new value in the update result), the update Result data structure needs to be changed. The current structure is based on file -> objects -> images. This only accounts for updates that involve the full image (name+tag) updates. Partial image updates (name or tag) are not recorded in the result. In order to provide complete details about what was updated, we need to record the complete change regardless of image reference. A new ResultV2 can be introduced to capture all the kinds of updates and the previous value with the structure file -> objects -> changes. A change constitutes the old value, the new value and the setter responsible for the update. This can be used to construct commit messages like the following with old and new values:

Automation: default/test-update-auto

- File: foo-deployment.yaml
  - Object: Deployment/default/foo
    Changes:
    - 2.2.2 -> 5.0.3
    
- File: podinfo-deployment.yaml
  - Object: Deployment/default/infopod
    Changes:
    - v1.0 -> 5.0.3
  - Object: Deployment/default/podinfo
    Changes:
    - ghcr.io/stefanprodan/podinfo:4.0.6 -> ghcr.io/stefanprodan/podinfo:5.0.3
    - bar -> ghcr.io/stefanprodan/podinfo
    - 4.0.6 -> 5.0.3

In order to keep the current commit templates working as before, TemplateData will be extended by introducing the new field for file change based data.

type TemplateData struct {
        AutomationObject types.NamespacedName
        Updated          update.Result
        Changed          update.ResultV2
}

The new result will be an opt-in feature.

See https://github.com/fluxcd/image-automation-controller/pull/642 for the implementation.

ImagePolicy selector support

Currently, all the ImagePolicies present in a namespace are considered when performing an update. Based on a lot of user feedback, policy selectors will be introduced will allow selecting image policies based on their labels via .spec.policySelector field in ImageUpdateAutomation. An implementation of this exists in https://github.com/fluxcd/image-automation-controller/pull/619. Along with this, the selected image policies will also be recorded in the status of ImageUpdateAutomation object with their latest images. This will help evaluate and reason about the result of an update, enhancing troubleshooting experience.

apiVersion: image.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
  name: update-app
spec:
  policySelector:
    matchLabels:
      app.kubernetes.io/component: foo
      app.kubernetes.io/instance: bar
  ...
status:
  observedPolicies:
    policyA:
      name: foo/bar1
      tag: v1.0
    policyB:
      name: foo/bar2
      tag: v1.5
  ...

short-circuit reconciliations

When nothing has changed (git repository commits and the image policies latest images), the reconciliations can be returned early, avoiding full reconciliations. If the latest images of the ImagePolicies have not changed and the remote git repository has no new commit, the full reconciliation operation can be skipped. This will require recording the ImagePolicies and their latest images observed in the last reconciliation, which is described above. And also recording the last observed revision in the git repository. We use similar technique in source-controller to avoid rebuilding the artifact if there's no new commit in the git repository. These can be stored in the status of the ImageUpdateAutomation, .status.observedPolicies and .status.observedSourceRevision.

status:
  ...
  lastPushCommit: ddaa3f4a10ec532c903890c6b51fdf64674df570
  observedPolicies:
    podinfo:
      name: ghcr.io/stefanprodan/podinfo
      tag: 5.0.3
    podinfo-test-1:
      name: ghcr.io/stefanprodan/podinfo
      tag: 4.0.6
    podinfo-test-2:
      name: ghcr.io/stefanprodan/podinfo
      tag: 4.0.0
  observedSourceRevision: main@sha1:ff3f5bd30ba6da07f5f6830eedc6bb62e471f8fb

During a reconciliation, git repository can be checked for new commit and the image policies can be checked for new latest images before performing the full reconciliation. The observedSourceRevision will follow any updates in the remote git repository and it will diverge from the lastPushCommit until new updates are pushed.

Limitation: When using a different push branch from the checkout branch or even a refspec, it's difficult to detect a drift in the remote branch at present. This short-circuit reconciliation will only apply when the push branch and checkout branch are the same. In the future, we can work on adding capabilities to also check push branch and refspec and extend this capability.

TODO:

• [x] Introduce ResultV2
• [ ] Introduce policy selector and observed policies
• [ ] short-circuit reconciliations with observed last commit and observed image policies