Image updater updates the wrong helm values when having more than one chart in an Argocd Application

[emagiz]

Describe the bug When having more than one helm chart deployed in the same Application file, the annotation argocd-image-updater.argoproj.io/XXXX.helm.image-name does not update the properties for the correct chart, but for the first one. To Reproduce Application.yaml

kind: Application
metadata:
  name: name
  namespace: namespace
  annotations:
    argocd-image-updater.argoproj.io/image-list: XXXXX=
    argocd-image-updater.argoproj.io/XXXXX.platforms: linux/amd64
    argocd-image-updater.argoproj.io/XXXXX.pull-secret: pullsecret:
    argocd-image-updater.argoproj.io/XXXXX.update-strategy: semver
    argocd-image-updater.argoproj.io/XXXXX.allow-tags: regexp:^[0-9]+
    argocd-image-updater.argoproj.io/XXXXX.ignore-tags: latest
    argocd-image-updater.argoproj.io/XXXXX.helm.image-tag: YYYY.image.tag #(YYYY should be the root yaml 
property in the values file of chart 2)
    argocd-image-updater.argoproj.io/XXXXX.helm.image-name: YYYY.image.repository
spec:
  project: 
  sources:
    - repoURL: ""
      path: main
      targetRevision: test
      helm:
        valueFiles:
          - $values-chart1/path
    - repoURL: ""
      targetRevision: test
      ref: values-chart1
    - repoURL: ""
      path: helm
      targetRevision: test
      helm:
        valueFiles:
          - $values-chart2/
    - repoURL: ""
      targetRevision: test
      ref: values-chart2
  destination:
    server: "https://kubernetes.default.svc"
    namespace: namespace
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Expected behavior

kind: Application
metadata:
  name: name
  namespace: namespace
  annotations:
    argocd-image-updater.argoproj.io/image-list: XXXXX=
    argocd-image-updater.argoproj.io/XXXXX.platforms: linux/amd64
    argocd-image-updater.argoproj.io/XXXXX.pull-secret: pullsecret:
    argocd-image-updater.argoproj.io/XXXXX.update-strategy: semver
    argocd-image-updater.argoproj.io/XXXXX.allow-tags: regexp:^[0-9]+
    argocd-image-updater.argoproj.io/XXXXX.ignore-tags: latest
    argocd-image-updater.argoproj.io/XXXXX.helm.image-tag: YYYY.image.tag #(YYYY should be the root
 yaml property in the values file of chart 2)
    argocd-image-updater.argoproj.io/XXXXX.helm.image-name: YYYY.image.repository
spec:
  project: 
  sources:
    - repoURL: ""
      path: main
      targetRevision: test
      helm:
        valueFiles:
          - $values-chart1/path
       # Actual property overwrite location 
       parameters:
          - name: YYYY.image.repository
            value: image
            forceString: true
          - name: YYYY.image.tag
            value: '1'
            forceString: true
    - repoURL: ""
      targetRevision: test
      ref: values-chart1
    - repoURL: ""
      path: helm
      targetRevision: test
      helm:
        valueFiles:
          - $values-chart2/
       # Expected property overwrite location 
       parameters:
          - name: YYYY.image.repository
            value: image
            forceString: true
          - name: YYYY.image.tag
            value: '1'
            forceString: true
    - repoURL: ""
      targetRevision: test
      ref: values-chart2
  destination:
    server: "https://kubernetes.default.svc"
    namespace: namespace
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Version Chart version 0.11.0

Logs The logs mentions that the properties are updated correctly and does not throw errors, but the place where the properties are updated is logically wrong

Additional-context In the case above, we only had to swap chart-2 with chart-1 in the Application.yaml and it worked fine, since for chart-1 we don't need to use image updater. But in the cases where we have to update two images in two different values files which are in the same Application.yaml, this will not succeed. (Splitting the charts to different apps is a workaround ofcourse, but not always desired)

[chengfang]

so the issue is, you're expecting the updates in repo1, but it actually goes to repo2 in this multi-source app?

Have you tried git-repository annotation to explicitly specify the target git repo (https://argocd-image-updater.readthedocs.io/en/latest/basics/update-methods/#specifying-a-repository-when-using-a-helm-repository-in-repourl)?

[tramiaczek]

I have exactly the same problem when using argocd-image-updater.argoproj.io/write-back-method: argocd (the default one), so in that case specifying git-repository does not help as I do not want to use that method. My application.yaml looks preety the same and works correctly, but I have to manualy delete the pod to force its image update.

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapps
  namespace: argocd
  annotations:
    argocd-image-updater.argoproj.io/image-list: >-
      app2=some.repo.address/repository/docker/snapshots/app2:latest,
      app1=some.repo.address/repository/docker/snapshots/app1:latest,
      app3=some.repo.address/repository/docker/snapshots/app3:latest
    argocd-image-updater.argoproj.io/app1.helm.image-tag: deployment.image.tag
    argocd-image-updater.argoproj.io/app1.update-strategy: digest
    argocd-image-updater.argoproj.io/app2.helm.image-tag: deployment.image.tag
    argocd-image-updater.argoproj.io/app2.update-strategy: digest
    argocd-image-updater.argoproj.io/app3.helm.image-tag: deployment.image.tag
    argocd-image-updater.argoproj.io/app3.update-strategy: digest

spec:
  destination:
    name: ''
    namespace: myapps
    server: 'https://kubernetes.default.svc'
  sources:
    - repoURL: 'https://some.repo.address/repository/helm'
      chart: app1
      targetRevision: 1.0.0
      helm:
        valueFiles:
          - $values/apps/dev/app1-values.yaml
    - repoURL: 'https://some.repo.address/repository/helm'
      chart: app2
      targetRevision: 1.0.2
      helm:
        valueFiles:
          - $values/apps/dev/app2-values.yaml
    - repoURL: 'https://some.repo.address/repository/helm'
      chart: app3
      targetRevision: 1.0.1
      helm:
        valueFiles:
          - $values/apps/devs/app3-values.yaml
    - repoURL: 'https://gitlab.address/devops/env.git'
      targetRevision: HEAD
      ref: values
  project: myapps
  syncPolicy:
    automated:
      prune: true
    syncOptions:
      - CreateNamespace=true

Image updater recognizes that the new image (with new digest) was pushed to the repository, but gets that digest and for example tries to update the first one deployment (argocd-image-updater.argoproj.io/app1.helm.image-tag) even if app3 image was pushed. It causes deployment errors as for app1 the image with digest: sha265:XYZ does not exists - this is digest for app3.

The message looks like: Failed to pull image "some.repo.address/repository/docker/snapshots/app1:latest@sha256:ed9f7fcdba20cb9c1cef4550f407422937896ddc97f3de834226ae0d1a6d3723": Error response from daemon: manifest for some.repo.address/repository/docker/snapshots/app1@sha256:ed9f7fcdba20cb9c1cef4550f407422937896ddc97f3de834226ae0d1a6d3723 not found: manifest unknown: manifest unknown

I know that as a workaround I could prepare single values.yaml file and specify the image.tag using app prefix, like: app1.image.tag and the use:

    argocd-image-updater.argoproj.io/app1.helm.image-tag: app1.deployment.image.tag
    argocd-image-updater.argoproj.io/app1.update-strategy: digest
    argocd-image-updater.argoproj.io/app2.helm.image-tag: app2.deployment.image.tag
    argocd-image-updater.argoproj.io/app2.update-strategy: digest
    argocd-image-updater.argoproj.io/app3.helm.image-tag: app3.deployment.image.tag
    argocd-image-updater.argoproj.io/app3.update-strategy: digest

but I would prefer to have separate values files to keep the possibility to deploy some subset of the apps, not always all of them. I do not want to prepare merged values file each time with double (or trippled) field names prefixed with app.

[etienne-napoleone]

This is a critical bug for me. I have an application composed of three Helm charts:

• API
• Poller
• Migration

The Migration component must be part of the same application because it includes a PreSync hook Job that scales down all replicas before running migrations. If it’s in a separate application, ArgoCD will immediately reconcile the replica count, preventing proper scaling down.

When new image versions are available (typically for all components at once), the API chart gets updated with the Poller image instead.

This seems like a major bug with potentially serious consequences, as it allows unintended containers from the same application to be injected into a deployment.

Manifest (it's a patch but has everything needed to understand the problem):

# app-patch.yaml
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: redacted
  namespace: argocd
  annotations:
    argocd-image-updater.argoproj.io/image-list: "api=ghcr.io/redacted/redacted-api:staging,poller=ghcr.io/redacted/redacted-poller:staging"
    argocd-image-updater.argoproj.io/api.update-strategy: digest
    argocd-image-updater.argoproj.io/api.helm.image-spec: image
    argocd-image-updater.argoproj.io/poller.update-strategy: digest
    argocd-image-updater.argoproj.io/poller.helm.image-spec: image
spec:
  sources:
    - repoURL: https://github.com/redacted/redacted.git
      path: charts/api
      targetRevision: staging
      helm:
        releaseName: redacted
        valuesObject:
          image: ghcr.io/redacted/redacted-api:staging
          imagePullPolicy: Always
          imagePullSecret: ghcr-secret
          envFrom:
            - secretRef:
                name: redacted-db-creds
    - repoURL: https://github.com/redacted/redacted.git
      path: charts/migration
      targetRevision: staging
      helm:
        releaseName: migration
        valuesObject:
          image: ghcr.io/redacted/redacted-migration:staging
          imagePullPolicy: Always
          imagePullSecret: ghcr-secret
          envFrom:
            - secretRef:
                name: redacted-db-creds
    - repoURL: https://github.com/redacted/redacted.git
      path: charts/poller
      targetRevision: staging
      helm:
        releaseName: initiation-2
        valuesObject:
          image: ghcr.io/redacted/redacted-poller:staging
          imagePullPolicy: Always
          imagePullSecret: ghcr-secret
          envFrom:
            - secretRef:
                name: redacted-db-creds
  syncPolicy:
    syncOptions:
      - CreateNamespace=true

Logs, it looks like it's saying it's updating the right thing:

level=info msg="Setting new image to ghcr.io/redacted/redacted-api:staging@sha256:1ae398caa77d31b04a2e3a1a7de4d9a79efa0409034a99bab1a030367f87c722" alias=api application=redacted image_name=redacted/redacted-api image_tag=dummy registry=ghcr.io
level=info msg="Successfully updated image 'ghcr.io/redacted/redacted-api:staging@dummy' to 'ghcr.io/redacted/multisig-api:staging@sha256:1ae398caa77d31b04a2e3a1a7de4d9a79efa0409034a99bab1a030367f87c722', but pending spec update (dry run=false)" alias=api application=multisig image_name=redacted/multisig-api image_tag=dummy registry=ghcr.io
level=info msg="Setting new image to ghcr.io/redacted/multisig-poller:staging@sha256:b2abbf471766d856d6012d383c018e74816c7da83583bb7122e1d00dda9e9616" alias=poller application=multisig image_name=redacted/multisig-poller image_tag=dummy registry=ghcr.io
level=info msg="Successfully updated image 'ghcr.io/redacted/redacted-poller:staging@dummy' to 'ghcr.io/redacted/redacted-poller:staging@sha256:b2abbf471766d856d6012d383c018e74816c7da83583bb7122e1d00dda9e9616', but pending spec update (dry run=false)" alias=poller application=redacted image_name=redacted/redacted-poller image_tag=dummy registry=ghcr.io
level=info msg="Committing 2 parameter update(s) for application redacted" application=redacted
level=info msg="Successfully updated the live application spec" application=redacted
level=info msg="Processing results: applications=1 images_considered=2 images_skipped=0 images_updated=2 errors=0"

But it updates the image for api with the poller image (updated manifest from argocd UI):

sources:
  - repoURL: https://github.com/redacted/redacted.git
    path: charts/api
    targetRevision: staging
    helm:
      parameters:
        - name: image
          value: >-
            ghcr.io/redacted/redacted-poller:staging@sha256:b2a101f0da48daa17514aaf7fda9e1ff565e757b5dbb58256d467ec32bd363db

There was a PR that was kind of ignored working on this I believe https://github.com/argoproj-labs/argocd-image-updater/pull/635