argo-cd icon indicating copy to clipboard operation
argo-cd copied to clipboard

[ApplicationSet/ProgressiveSync] removing resources triggers an infinite sync loop

Open skwair opened this issue 3 weeks ago • 6 comments

Checklist:

  • [x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
  • [x] I've included steps to reproduce the bug.
  • [x] I've pasted the output of argocd version.

Describe the bug

Deleting a resource in an Application generated by an ApplicationSet with Progressive Syncs enabled triggers an infinite sync loop.

In the following clip, I initially have 1 deployment and 2 configmaps in my Application. I refresh the application to a new commit, removing one configmap:

https://github.com/user-attachments/assets/6f911538-a111-441f-9f38-bdf390f839a2

To Reproduce

  1. Apply the following ApplicationSet:
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: 'app'
  namespace: 'argocd'
spec:
  goTemplate: true
  goTemplateOptions: [ 'missingkey=error' ]
  generators:
    - git:
        repoURL: 'https://github.com/skwair/argocd-issue-25444.git'
        revision: 'HEAD'
        directories:
          - path: 'overlays/*'
  template:
    metadata:
      name: 'app-{{ index .path.segments 1 }}'
      labels:
        env: 'prod'
    spec:
      project: 'default'
      source:
        repoURL: 'https://github.com/skwair/argocd-issue-25444.git'
        targetRevision: 'HEAD'
        path: '{{ .path.path }}'
      destination:
        server: 'https://kubernetes.default.svc'
        namespace: 'app-{{ index .path.segments 1 }}'
      syncPolicy:
        syncOptions:
          - 'CreateNamespace=true'
          - 'ServerSideApply=true'
  strategy:
    type: 'RollingSync'
    rollingSync:
      steps:
        - matchExpressions:
            - key: 'env'
              operator: 'In'
              values: ['prod']
          maxUpdate: 1
  1. Remove any resource from any generated Application in Git
  2. Commit and push
  3. Trigger a sync via the UI (or wait for applications to refresh)

Expected behavior

To be honest, I'm not 100% sure what to expect since the RollingSync documentation states:

RollingSync will force all generated Applications to have autosync disabled.

Does it include automatic pruning?

Either way, if documentation implies automatic pruning is also disabled, I would expect the Application to simply sync once and mark deleted resource as deleted in Git, but kept in Kubernetes. Otherwise, I would expect Argo CD to prune these resources.

Key observations:

  • By default, prune seems to be disabled
  • Removing a resource triggers an infinite sync loop (the bug)
  • Explicitly setting automated.prune: true (despite documentation mentioning automated is disabled when using Progressive Syncs) on generated Applications fixes the issue
  • Similarly, adding the PruneLast=true sync option on generated Applications fixes the issue
  • Manually syncing (via the UI or using the CLI) with the prune option also works
  • selfHeal seems to always be active and works as expected

Is the official way of enabling automatic resource pruning when using Progressive Syncs setting either automated.prune: true or PruneLast=true?

Version

argocd: v3.2.0+66b2f30
  BuildDate: 2025-11-04T19:52:35Z
  GitCommit: 66b2f302d91a42cc151808da0eec0846bbe1062c
  GitTreeState: clean
  GoVersion: go1.25.3
  Compiler: gc
  Platform: darwin/arm64
argocd-server: v3.2.0+66b2f30
  BuildDate: 2025-11-04T14:51:35Z
  GitCommit: 66b2f302d91a42cc151808da0eec0846bbe1062c
  GitTreeState: clean
  GoVersion: go1.25.0
  Compiler: gc
  Platform: linux/arm64
  Kustomize Version: v5.7.0 2025-06-28T07:00:07Z
  Helm Version: v3.18.4+gd80839c
  Kubectl Version: v0.34.0
  Jsonnet Version: v0.21.0

Logs

These are the logs that get infinitely spammed when the sync is triggered:

{"applicationset":{"Namespace":"argocd","Name":"app"},"level":"info","msg":"generated 3 applications","time":"2025-11-29T10:50:28Z"}
{"applicationset":{"Namespace":"argocd","Name":"app"},"level":"info","msg":"ApplicationSet app step list:","time":"2025-11-29T10:50:28Z"}
{"applicationset":{"Namespace":"argocd","Name":"app"},"level":"info","msg":"step 1: [app-tenant1 app-tenant2 app-tenant3]","time":"2025-11-29T10:50:28Z"}
{"applicationset":{"Namespace":"argocd","Name":"app"},"level":"info","msg":"Application allowed to sync before maxUpdate?: map[app-tenant1:true app-tenant2:true app-tenant3:true]","time":"2025-11-29T10:50:28Z"}
{"applicationset":{"Namespace":"argocd","Name":"app"},"level":"info","msg":"end reconcile in 316.122392ms","requeueAfter":180000000000,"time":"2025-11-29T10:50:28Z"}
{"allPaths":["base","overlays","overlays/tenant1","overlays/tenant2","overlays/tenant3"],"level":"info","msg":"applications result from the repo service","pathParamPrefix":"","repoURL":"https://github.com/skwair/argocd-issue-25444.git","revision":"HEAD","time":"2025-11-29T10:53:28Z","total":5}

skwair avatar Nov 29 '25 11:11 skwair

I've made a minimal public repository to illustrate the bug: https://github.com/skwair/argocd-issue-25444

Steps to reproduce using this repo:

  1. Apply the initial ApplicationSet (kubectl apply -f application-set.yaml)
  2. Comment base/kustomization.yaml#L6 for example
  3. Commit & push
  4. Refresh an app -> it'll sync indefinitely
  5. Un-comment application-set.yaml#L33 or application-set.yaml#L34-L35 and re-apply the ApplicationSet to resolve the issue

Let me know if you need further information!

skwair avatar Nov 29 '25 11:11 skwair

Has anyone picked this up, or can I look into it?

fsdkibe avatar Nov 30 '25 21:11 fsdkibe

@fsdkibe feel free to take this up. @skwair the repository that you've referenced in your appset manifest doesn't exist or maybe it's private, can you change it to public so anyone who tries to reproduce this issue can easily do it?

nitishfy avatar Dec 01 '25 06:12 nitishfy

tagging @ranakan19 in case she has any input into it.

nitishfy avatar Dec 01 '25 06:12 nitishfy

@nitishfy the public repo is in the comment bellow, but I've edited the main issue description so it now also points to the same repo!

Thanks for taking a look into it!

skwair avatar Dec 01 '25 07:12 skwair

Duly noted @nitishfy.

fsdkibe avatar Dec 01 '25 14:12 fsdkibe