argocd application do not update when values file changed in helm repo.

[hillbun]

argocd v2.1.7

When I create application connect to helm repo, when values file in helm repo is changed, application do not update, even I click sync or refresh.

Application only changed when I click hard refresh.

How to solve it?

[Terry-Yi]

I also encountered such problems, and I also found that the same version of the chart, such as adding another value file values-poc.yaml, when valueFiles is values-poc.yaml, will report an error values-poc.yaml during deployment: no such file or directory

[vvatlin]

Looks like I have the same issue my application.yaml looks like this:

kind: Application
metadata:
  name: istiod
  namespace: argocd
spec:
  source:
    targetRevision: 1.13.3
    helm:
      values: |-
        global:
          hub: gcr.io/istio-release
          tag: 1.13.3
        meshConfig:
          accessLogFile: /dev/stdout
          ingressService: istio-ingress
          ingressSelector: ingress

When I change values for helm chart nothing happen, even if I do sync, I still have the same old ConfigMap for Istio.

[mbaldessari]

I have observed this as well. From my initial cursory testing it seems that a hard-refresh fixes things. E.g.:

argocd app get test --hard-refresh

FWIW I just tested with v2.4.0-rc5+b84dd8b and I hit it there as well

[EnriqueHormilla]

Same issues here, I have Argo CD : v2.0.5 When I update some setting in my values.yaml I can see are updated inside the app, but don't run any update. That confirm me Argo read the correct values.yaml I tried to do hard refresh, but don't work. After that I remove one deployment object and did a rsync, the object was created but don't update the settings.

I feel it's relational with number of "rev:1" like I need to increase the number of revision in some way.

[crenshaw-dev]

Does anyone have a public repo I can test against?

I just tried with 2.4.2, and changing replicaCount was reflected in the UI with a simple "Refresh." https://github.com/crenshaw-dev/argocd-example-apps/commits/values-change

(Aside @EnriqueHormilla and @hillbun I'd upgrade ASAP if your instance is anything other than a test instance, there are 11 CVEs against 2.0.5 and 2.1.7.)

[EnriqueHormilla]

Hi @crenshaw-dev thanks for the support. I know the change is reflected in the UI, but despite that, argoCD doesn't do anything with this change. In this picture, i have configured argo in my repo like that:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: apache-custom
  namespace: argocd
spec:
  destination:
    namespace: apache-custom
    server: https://kubernetes.default.svc
  project: default
  source:
    helm:
      valueFiles:
      - values-stg.yaml
    path: path/apache-custom
    repoURL: [email protected]:privateRepo/repo.git
    targetRevision: test

In my values-stg.yaml I have replicaCount :6, but I have only one pod.

If I configured Argo directly to helm repo:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: demo
  namespace: argocd
spec:
  project: default
  source:
    chart: apache
    helm:
      parameters:
        - name: replicaCount
          value: '6'
    repoURL: 'https://charts.bitnami.com/bitnami'
    targetRevision: 9.1.11        
  destination:
    server: https://kubernetes.default.svc
    namespace: apache-custom
  syncPolicy:
    automated: {}

Its work and the deployment increase the pods.

[EnriqueHormilla]

Hi @crenshaw-dev finally I did the workaround suggest for the community in https://github.com/argoproj/argo-cd/issues/2789#issuecomment-1167345080 .

[crenshaw-dev]

@EnriqueHormilla can you clarify what you mean by "argoCD doesn't do anything with this change"? When I change the replicaCount, not only is it reflected in the UI, but when I sync the replicaCount is matched by the number of pods.

The plugin workaround should be unnecessary. Generally I avoid plugin hacks when the built-in config management tool can do the job. Plugins are more difficult to secure, and you may have to re-implement features that are already baked into the default implementation.

[EnriqueHormilla]

Hi @crenshaw-dev, I agree with you to try to avoid used custom plugins, but in the tests I did don't work. I want to have a custom external value file for my helm chart (don't use the value.yaml from helm repo), I don't want to write in the Argo app definition my custom values as second example. I want as the first example. ArgoCD can see the values-stg.yaml and see the changes, but these changes aren't reflected in the app. No matter what I change, argoCD don't redeploy the app. In the before example both have the same custom values, (replicaCount) and as you can see the first don't “read” the value and don't add more pods.

Example: I change replica Count to 6, I can see in the argoCD GUI replicaCount6, but argoCD don't create more pods.

[crenshaw-dev]

Ah, that use case will require #2789 to be resolved. There's currently a draft PR, and it's planned for 2.5 in August.

[EnriqueHormilla]

Ah, that use case will require #2789 to be resolved. There's currently a draft PR, and it's planned for 2.5 in August.

Thanks @crenshaw-dev , good news!

[hillbun]

@crenshaw-dev

current release is v2.4.11, so this bug is not fixed yet?

[xgt001]

I am grateful to have found this thread as I was going a bit crazy thinking I am doing something somehow wrong. 😄 Can confirm in our own setup that only values.yaml is read and it doesnt bother reading even from inline values: as such

  source:
    repoURL: [email protected]/helm.git
    path: kubernetes/helm/bootstrap
    helm:
      #the below values are merrily ignored
      values: |
        testFeature:
          enabled: true
          overlay: non-production

Edit: kubectl apply -f application.yaml works. will use that as a workaround until the fix is released :)

[maaft]

@xgt001 what exactly is in application.yaml in your case?

I've installed a helm chart from a public repository. Now I want to change values.yaml and expect that argocd re-deploys pods, which is not the case.

[duxing]

@crenshaw-dev I'm also running into this issue.

I'm running [email protected]+91aefab

argocd: v2.4.0+91aefab
  BuildDate: 2022-06-10T17:23:37Z
  GitCommit: 91aefabc5b213a258ddcfe04b8e69bb4a2dd2566
  GitTreeState: clean
  GoVersion: go1.18.3
  Compiler: gc
  Platform: linux/amd64
argocd-server: v2.4.0+91aefab
  BuildDate: 2022-06-10T17:23:37Z
  GitCommit: 91aefabc5b213a258ddcfe04b8e69bb4a2dd2566
  GitTreeState: clean
  GoVersion: go1.18.3
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: v4.4.1 2021-11-11T23:36:27Z
  Helm Version: v3.8.1+g5cb9af4
  Kubectl Version: v0.23.1
  Jsonnet Version: v0.18.0

and the application configuration yaml (datadog helm chart in this case):

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: datadog
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: "default"
  source:
    repoURL: https://helm.datadoghq.com
    targetRevision: 3.1.3
    chart: datadog
    helm:
      values: |
        <content-of-my-values.yaml>
  destination:
    server: https://kubernetes.default.svc
    namespace: datadog
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
    retry:
      limit: 1
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 1m

when I modified the content inside the content-of-my-values.yaml block (.spec.source.helm.values) the app is marked as "synced" but the last applied configuration does not pick up my changes.

I checked argocd app get datadog -oyaml and .status.history[0] has the correct timestamp (days after I pushed a removal-only configuration change) but .status.history[0].source.helm.values was still using the revision before the push.

I noticed repo-server logs mentioning cache hit but I guess that's for the chart repo, which doesn't affect this. That lead me to believe the argocd server is responsible for the stale copy of the configuration.

[duxing]

the correct fix has been postponed to v2.6 (reference)

an alternative solution: https://github.com/argoproj/argo-cd/issues/2789#issuecomment-1176820186

[emmercm]

I believe I'm still experiencing this issue with v2.6.3 using in-line Helm values similar to:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: <redacted>
  namespace: argo-cd-dev
spec:
  project: <redacted>
  source:
    repoURL: <redacted>
    targetRevision: <redacted>
    chart: <redacted>
    helm:
      values: |
        server:
          replicaCount: 0
        # Many more lines of deeply-nested YAML...

  destination:
    name: k8s-dev
    namespace: <redacted>
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - PrunePropagationPolicy=background

Editing spec.source.helm.values in either Git (application is set to auto sync) or the Argo UI does not cause the replica count of my deployment to change. Initiating a forceful sync via the Argo UI afterward does cause the replica count to change.

[maaft]

@emmercm Until this is solved, here's what I did:

• create extra repo with your helm-chart
• if you have any 3rd party charts, use helm dependencies
• update values.yaml in that repository directly and argocd will correctly update deployements

[migueleliasweb]

I just wanted to leave an idea here as well. The company I work for, we were definitely stuck with this problem for a while now.

The way we we've fixed it was to completely ditch the native helm integration and create our own plugin (https://argo-cd.readthedocs.io/en/stable/operator-manual/config-management-plugins/). This plugin has some smarts around to fix some of the use cases we needed. E.g: Helm recursive dependencies, multi values files, multiple sources, mixed applications with Helm + Kustomize, etc.

I reckon if you're not afraid to write software, that is a much more complete solution instead of waiting for the open source to catch up and fix all your edge cases.

[Dreamystify]

Not sure if this is relevant to anyone but was the fix I had for my situation. I created a sub chart that had prometheus as a dependency. I had a values file with the values from the prometheus chart with my sub chart. The values would never update, they would always have the prometheus defaults. I had configured the values file wrong for the dependency chart. the prometheus chart needs to have its values in the sub-chart values file formatted like so:

prometheus:
   <prometheus chart values>:
<another dependency chart>:
   <values>...

After that they updated as expected

[LukePavelka]

this is still problem :(

[simonloach]

Hitting this issue as well right now.

[galaxyware241]

same issue here for us too.

[crenshaw-dev]

For the life of me, I cannot reproduce this. When I change the values, I immediately see the app go out of sync.

https://github.com/argoproj/argo-cd/assets/350466/8338e124-d9c3-4dec-bf28-500eebe8e6fb

Same when I use a Helm repo instead of a Helm chart hosted in git.

https://github.com/argoproj/argo-cd/assets/350466/b3639302-6193-4643-87a1-8e7790d41c94

[dglumac-nex]

Here's another little use case to try and help diagnose this.

I am experiencing this problem in ArgoCD 2.8.2, and in my case the values is updating a Job which has already completed. I tried various options like forcing a Replace=true on the sync-option annotations, I have even gone as far as naming the job with something from the values.yaml file.

Set-up:

• ArgoCD application monitors a path in another GitHub repository on the main branch
• The monitored path has a Helm chart with a single dependency and nothing else - in effect it just deploys that dependency
• Dependency chart has, among other things, a Job which changes the name based on a value in its values.yaml file
• I override the said value in the values.yaml file and push it to the main branch of the monitored GitHub repo
• ArgoCD is set up to auto sync with defaults (3 mins poll time etc)

What happens:

• ArgoCD eventually detects that there has been a change to the path of the monitored repository, it updates the commit reference and the comment in the Sync Status section - second from the left in the ribbon at the top
• The Last Sync section - the third form the left in the ribbon at the top - still shows the previous commit reference and the previous commit message
• Sync Status section shows Synced status (to the latest commit)
• Last Sync section shows Sync OK status (to the previous commit)

Now I click on the Sync button.

The Sync popup overlay shows all other resources in the Synchronize Resources section at the bottom except the Job which I am trying to update/replace.

I wonder if it has something to do with the fact that the Job has completed and is now no longer seen as "updateable".

But, when I then click on Synchronize (which I presume triggers a hard sync) the new commit is actually deployed, and the replacement job does appear and run.

I hope that helps.

UPDATE

Yes, I think I may be onto something here.

I have also updated a config map from the same dependency Helm chart to have an annotation that includes the value I am updating, and that triggers a resync and correctly creates a new job.

[kvaps]

Hi all, I made another workaround for this:

server:
  extraContainers:
  - name: argocd-hard-refresher
    image: bitnami/kubectl:1.26.9
    command:
    - /bin/sh
    - -c
    args:
      - |
        while kubectl annotate application.argoproj.io -A -l hard-refresh=true argocd.argoproj.io/refresh=hard --overwrite >/dev/null; do
          sleep 60
        done

Now any app that has hard-refresh=true label with be hard refreshed for every 60 seconds

[ionutz89]

Is it any update about this issue?

[tbotnz]

still having this issue

[mpycio]

really weird, we've been using argocd for almost a year with no sync issues, sometimes I had to use hard refresh. Today came across this issue, for no apparent reason it just won't recognise new image version I'm trying to push - UI shows all in sync and displays correct commit sha but uses old version.

[skhtor]

I've also just suddenly started experiencing this issue today,

It was fine last week, It was just as I went to demo Argo to my team when it started breaking, sorry all, looks like the Demo Gods have broken Argo for everybody.