applicationset icon indicating copy to clipboard operation
applicationset copied to clipboard

Published 20 hours ago • verified publisher icon argoproj

ApplicationSet Controller pulls git repos multiple times per minute instead of every 3m

Open dan1el-k opened this issue 2 years ago • 1 comments

We are running the ApplicationSet Controller Version v0.3.0 together with ArgoCd 2.1.6 (deployed via ArgoCD Operator) on multiple OKD clusters

We experienced that the controller for any reason is fetching more often that every 3 min So once every 1 min, or sometimes even every 20-30 seconds We have 5 different Application Sets running, all behaving in this same schedule. Multiplied this scenario with 10 OKD clusters, together with the standard ArgoCD sync behaviour, we are flooding our internal GitHub Enterprise with up to 100 fetch requests per seconds for some of the repos...

In all ApplicationSets, we are having

  • 1 Matrix generator
    • git generator
    • list generator
  • 1 git generator Both git generators are reading from the same repo.
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: "dev01-app-catalog"
spec:
  generators:
    #Teams Kustomize Application generators
    #Matrix generator currently not support template override !!!
    - matrix:
        generators:
          - list:
              elements:
                - environmentName: dev01
                  cortexClusterName: cluster01
                  account-testName: account-test
          - git:
              repoURL: https://code.my.domain/myorg/cortex-bootstrap-app-catalog.git
              revision: account-test/cluster01
              files:
              - path: 'environments/dev01/kustomize/*.yaml'
    #Teams Helm Application generators
    - git:
        repoURL: https://code.my.domain/myorg/cortex-bootstrap-app-catalog.git
        revision: account-test/cluster01
        files:
        - path: "environments/dev01/helm/*.yaml"
        template:
          metadata: {}
          spec:
            project: '{{project}}'
            source:
              targetRevision: '{{targetRevision}}'
              repoURL: '{{repoURL}}'
              path: deployment
              helm:
                valueFiles:
                  - values.yaml
                  - 'accounts/account-test/values.yaml'
                  - 'clusters/cluster01/values.yaml'
                  - 'environments/dev01/values.yaml'
                  - 'stages/account-test-cluster01-dev01/values.yaml'
            destination: {}
  template:
    metadata:
      name: 'dev01-{{name}}'
    spec:
      project: '{{project}}'
      source:
        targetRevision: '{{targetRevision}}'
        repoURL: '{{repoURL}}'
        path: 'deployment/stages/{{accountName}}-{{cortexClusterName}}-{{environmentName}}'
      destination:
        name: cortex
        namespace: 'dev01-{{namespace}}'
      syncPolicy:
        syncOptions:
        - CreateNamespace=true
        - ApplyOutOfSyncOnly=true
        retry:
          limit: 3 # number of failed sync attempt retries; unlimited number of attempts if less than 0
          backoff:
            duration: 5s # the amount to back off. Default unit is seconds, but could also be a duration (e.g. "2m", "1h")
            factor: 2 # a factor to multiply the base duration after each failed retry
            maxDuration: 2m # the maximum amount of time allowed for the backoff strategy
        automated:
          prune: true
          selfHeal: true

Here the logs with the behaviour

I0201 20:12:33.587484       1 request.go:665] Waited for 1.024107965s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/machineconfiguration.openshift.io/v1?timeout=32s
time="2022-02-01T20:12:36Z" level=info msg="Starting configmap/secret informers"
time="2022-02-01T20:12:36Z" level=info msg="Configmap/secret informer synced"
time="2022-02-01T20:12:36Z" level=info msg="processing event for cluster secret" name=argocd-cortex-cluster-config namespace=cortex-argocd type=createSecretEventHandler
time="2022-02-01T20:12:36Z" level=info msg="listed ApplicationSets" count=5 type=createSecretEventHandler
time="2022-02-01T20:12:36Z" level=info msg="processing event for cluster secret" name=argocd-default-cluster-config namespace=cortex-argocd type=createSecretEventHandler
time="2022-02-01T20:12:36Z" level=info msg="listed ApplicationSets" count=5 type=createSecretEventHandler
time="2022-02-01T20:12:36Z" level=info msg="Initializing https://code.rbi.tech/raiffeisen/cortex-bootstrap-app-catalog.git to /tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog"
time="2022-02-01T20:12:36Z" level=info msg="rm -rf /tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog" dir= execID=619ae
time="2022-02-01T20:28:15Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=ca52d
time="2022-02-01T20:28:15Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=438.698765
time="2022-02-01T20:28:15Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=3094c
time="2022-02-01T20:28:15Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.532365
time="2022-02-01T20:28:15Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=a8d2a
time="2022-02-01T20:28:15Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.142427
time="2022-02-01T20:28:15Z" level=info msg="git ls-files --full-name -z -- community/dev01/kustomize/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=61864
time="2022-02-01T20:28:15Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/kustomize/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.1320509999999997
time="2022-02-01T20:28:15Z" level=info msg="generated 0 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc0011fc000 <nil>}"
time="2022-02-01T20:28:15Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=9e9bf
time="2022-02-01T20:28:15Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=387.995245
time="2022-02-01T20:28:16Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=cd42f
time="2022-02-01T20:28:16Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=22.072674
time="2022-02-01T20:28:16Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=ed051
time="2022-02-01T20:28:16Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=10.76123
time="2022-02-01T20:28:16Z" level=info msg="git ls-files --full-name -z -- community/dev01/helm/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=5350f
time="2022-02-01T20:28:16Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/helm/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=5.5571839999999995
time="2022-02-01T20:28:16Z" level=info msg="generated 4 applications" generator="{<nil> <nil> 0xc000e56d00 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-01T20:28:16Z" level=info msg="unchanged Application" app=dev01-neuron-pipelinefunctions appSet=community-dev01-app-catalog
time="2022-02-01T20:28:16Z" level=info msg="unchanged Application" app=dev01-neuron-pulsar appSet=community-dev01-app-catalog
time="2022-02-01T20:28:16Z" level=info msg="unchanged Application" app=dev01-neuron-superset appSet=community-dev01-app-catalog
time="2022-02-01T20:28:16Z" level=info msg="unchanged Application" app=dev01-neuron-trino appSet=community-dev01-app-catalog
time="2022-02-01T20:28:16Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-01T20:29:17Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=36d2b
time="2022-02-01T20:29:18Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=671.824143
time="2022-02-01T20:29:18Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=7b94b
time="2022-02-01T20:29:18Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=18.008559
time="2022-02-01T20:29:18Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=9f185
time="2022-02-01T20:29:18Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=23.961217
time="2022-02-01T20:29:18Z" level=info msg="git ls-files --full-name -z -- community/dev01/kustomize/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=923a7
time="2022-02-01T20:29:18Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/kustomize/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=15.895937999999997
time="2022-02-01T20:29:18Z" level=info msg="generated 0 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc001332000 <nil>}"
time="2022-02-01T20:29:18Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=77c84
time="2022-02-01T20:29:19Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=578.44983
time="2022-02-01T20:29:19Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=b27f1
time="2022-02-01T20:29:19Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=5.8432889999999995
time="2022-02-01T20:29:19Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=fe117
time="2022-02-01T20:29:19Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.4708479999999997
time="2022-02-01T20:29:19Z" level=info msg="git ls-files --full-name -z -- community/dev01/helm/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=a8183
time="2022-02-01T20:29:19Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/helm/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.615574
time="2022-02-01T20:29:19Z" level=info msg="generated 4 applications" generator="{<nil> <nil> 0xc00090c680 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-01T20:29:19Z" level=info msg="unchanged Application" app=dev01-neuron-pipelinefunctions appSet=community-dev01-app-catalog
time="2022-02-01T20:29:19Z" level=info msg="unchanged Application" app=dev01-neuron-pulsar appSet=community-dev01-app-catalog
time="2022-02-01T20:29:19Z" level=info msg="unchanged Application" app=dev01-neuron-superset appSet=community-dev01-app-catalog
time="2022-02-01T20:29:19Z" level=info msg="unchanged Application" app=dev01-neuron-trino appSet=community-dev01-app-catalog
time="2022-02-01T20:29:19Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-01T20:30:02Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=3ab95
time="2022-02-01T20:30:02Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=440.451326
time="2022-02-01T20:30:02Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=8ebe5
time="2022-02-01T20:30:02Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.7640949999999997
time="2022-02-01T20:30:02Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=3a8ac
time="2022-02-01T20:30:02Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.0786670000000003
time="2022-02-01T20:30:02Z" level=info msg="git ls-files --full-name -z -- community/dev01/kustomize/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=48f13
time="2022-02-01T20:30:02Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/kustomize/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.491303
time="2022-02-01T20:30:02Z" level=info msg="generated 0 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc001400580 <nil>}"
time="2022-02-01T20:30:02Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=36142
time="2022-02-01T20:30:03Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=438.434581
time="2022-02-01T20:30:03Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=bc71a
time="2022-02-01T20:30:03Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.880461
time="2022-02-01T20:30:03Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=ca764
time="2022-02-01T20:30:03Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.61766
time="2022-02-01T20:30:03Z" level=info msg="git ls-files --full-name -z -- community/dev01/helm/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=f1a51
time="2022-02-01T20:30:03Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/helm/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.19903
time="2022-02-01T20:30:03Z" level=info msg="generated 4 applications" generator="{<nil> <nil> 0xc00090c1a0 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-01T20:30:03Z" level=info msg="unchanged Application" app=dev01-neuron-pipelinefunctions appSet=community-dev01-app-catalog
time="2022-02-01T20:30:03Z" level=info msg="unchanged Application" app=dev01-neuron-pulsar appSet=community-dev01-app-catalog
time="2022-02-01T20:30:03Z" level=info msg="unchanged Application" app=dev01-neuron-superset appSet=community-dev01-app-catalog
time="2022-02-01T20:30:03Z" level=info msg="unchanged Application" app=dev01-neuron-trino appSet=community-dev01-app-catalog
time="2022-02-01T20:30:03Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-01T20:30:14Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=1fcf6
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=387.629545
time="2022-02-01T20:30:15Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=504b5
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.458901
time="2022-02-01T20:30:15Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=8b362
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.9481920000000001
time="2022-02-01T20:30:15Z" level=info msg="git ls-files --full-name -z -- community/dev01/kustomize/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=d0ba1
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/kustomize/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.2898930000000002
time="2022-02-01T20:30:15Z" level=info msg="generated 0 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000dc8160 <nil>}"
time="2022-02-01T20:30:15Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=c903a
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=369.431777
time="2022-02-01T20:30:15Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=c17b7
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.44612
time="2022-02-01T20:30:15Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=10b7e
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.18178
time="2022-02-01T20:30:15Z" level=info msg="git ls-files --full-name -z -- community/dev01/helm/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=14007
time="2022-02-01T20:30:15Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/helm/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.273115
time="2022-02-01T20:30:15Z" level=info msg="generated 4 applications" generator="{<nil> <nil> 0xc00090d040 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-01T20:30:15Z" level=info msg="unchanged Application" app=dev01-neuron-pipelinefunctions appSet=community-dev01-app-catalog
time="2022-02-01T20:30:15Z" level=info msg="unchanged Application" app=dev01-neuron-pulsar appSet=community-dev01-app-catalog
time="2022-02-01T20:30:15Z" level=info msg="unchanged Application" app=dev01-neuron-superset appSet=community-dev01-app-catalog
time="2022-02-01T20:30:15Z" level=info msg="unchanged Application" app=dev01-neuron-trino appSet=community-dev01-app-catalog
time="2022-02-01T20:30:15Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-01T20:30:45Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=e22cf
time="2022-02-01T20:30:47Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2327.068889
time="2022-02-01T20:30:47Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=fd97f
time="2022-02-01T20:30:47Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=117.281728
time="2022-02-01T20:30:47Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=9239d
time="2022-02-01T20:30:47Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=17.547248
time="2022-02-01T20:30:47Z" level=info msg="git ls-files --full-name -z -- community/dev01/kustomize/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=658c9
time="2022-02-01T20:30:47Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/kustomize/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.4767449999999998
time="2022-02-01T20:30:47Z" level=info msg="generated 0 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000e90dc0 <nil>}"
time="2022-02-01T20:30:47Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=e9cd5
time="2022-02-01T20:30:48Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1070.9273269999999
time="2022-02-01T20:30:49Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=78744
time="2022-02-01T20:30:49Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.88776
time="2022-02-01T20:30:49Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=1737a
time="2022-02-01T20:30:49Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.117585
time="2022-02-01T20:30:49Z" level=info msg="git ls-files --full-name -z -- community/dev01/helm/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=e15f0
time="2022-02-01T20:30:49Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/helm/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.590189
time="2022-02-01T20:30:49Z" level=info msg="generated 4 applications" generator="{<nil> <nil> 0xc00026f380 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-01T20:30:49Z" level=info msg="unchanged Application" app=dev01-neuron-pipelinefunctions appSet=community-dev01-app-catalog
time="2022-02-01T20:30:49Z" level=info msg="unchanged Application" app=dev01-neuron-pulsar appSet=community-dev01-app-catalog
time="2022-02-01T20:30:49Z" level=info msg="unchanged Application" app=dev01-neuron-superset appSet=community-dev01-app-catalog
time="2022-02-01T20:30:49Z" level=info msg="unchanged Application" app=dev01-neuron-trino appSet=community-dev01-app-catalog
time="2022-02-01T20:30:49Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-01T20:31:12Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=6b64d
time="2022-02-01T20:31:13Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=615.8336770000001
time="2022-02-01T20:31:13Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=7674a
time="2022-02-01T20:31:13Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=14.172065
time="2022-02-01T20:31:13Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=2a953
time="2022-02-01T20:31:13Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.696586
time="2022-02-01T20:31:13Z" level=info msg="git ls-files --full-name -z -- community/dev01/kustomize/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=c9bcc
time="2022-02-01T20:31:13Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/kustomize/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.875745
time="2022-02-01T20:31:13Z" level=info msg="generated 0 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc0014f2dc0 <nil>}"
time="2022-02-01T20:31:13Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=c4f15
time="2022-02-01T20:31:14Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=693.4795839999999
time="2022-02-01T20:31:14Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=4d335
time="2022-02-01T20:31:14Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=13.193836000000001
time="2022-02-01T20:31:14Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=f11a5
time="2022-02-01T20:31:14Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=8.911314
time="2022-02-01T20:31:14Z" level=info msg="git ls-files --full-name -z -- community/dev01/helm/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=c9dde
time="2022-02-01T20:31:14Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/helm/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=12.894156
time="2022-02-01T20:31:14Z" level=info msg="generated 4 applications" generator="{<nil> <nil> 0xc001280820 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-01T20:31:14Z" level=info msg="unchanged Application" app=dev01-neuron-pipelinefunctions appSet=community-dev01-app-catalog
time="2022-02-01T20:31:14Z" level=info msg="unchanged Application" app=dev01-neuron-pulsar appSet=community-dev01-app-catalog
time="2022-02-01T20:31:14Z" level=info msg="unchanged Application" app=dev01-neuron-superset appSet=community-dev01-app-catalog
time="2022-02-01T20:31:14Z" level=info msg="unchanged Application" app=dev01-neuron-trino appSet=community-dev01-app-catalog
time="2022-02-01T20:31:14Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-01T20:31:18Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=00f14
time="2022-02-01T20:31:19Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=965.670647
time="2022-02-01T20:31:19Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=7d313
time="2022-02-01T20:31:19Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=5.03759
time="2022-02-01T20:31:19Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=312e0
time="2022-02-01T20:31:19Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.1076620000000004
time="2022-02-01T20:31:19Z" level=info msg="git ls-files --full-name -z -- community/dev01/kustomize/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=4b625
time="2022-02-01T20:31:19Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/kustomize/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.661803
time="2022-02-01T20:31:19Z" level=info msg="generated 0 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000d606e0 <nil>}"
time="2022-02-01T20:31:19Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=53af1
time="2022-02-01T20:31:20Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=703.2269570000001
time="2022-02-01T20:31:20Z" level=info msg="git checkout --force b7549fde938a7a1f476f50319effb084f260708c" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=9f370
time="2022-02-01T20:31:20Z" level=info msg=Trace args="[git checkout --force b7549fde938a7a1f476f50319effb084f260708c]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=10.288931
time="2022-02-01T20:31:20Z" level=info msg="git clean -fdx" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=069f3
time="2022-02-01T20:31:20Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=23.175719
time="2022-02-01T20:31:20Z" level=info msg="git ls-files --full-name -z -- community/dev01/helm/*.yaml" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog execID=f8d39
time="2022-02-01T20:31:20Z" level=info msg=Trace args="[git ls-files --full-name -z -- community/dev01/helm/*.yaml]" dir=/tmp/https___code.my.domain_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=53.359643999999996
time="2022-02-01T20:31:20Z" level=info msg="generated 4 applications" generator="{<nil> <nil> 0xc000b24000 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-01T20:31:20Z" level=info msg="unchanged Application" app=dev01-neuron-pipelinefunctions appSet=community-dev01-app-catalog
time="2022-02-01T20:31:20Z" level=info msg="unchanged Application" app=dev01-neuron-pulsar appSet=community-dev01-app-catalog
time="2022-02-01T20:31:20Z" level=info msg="unchanged Application" app=dev01-neuron-superset appSet=community-dev01-app-catalog
time="2022-02-01T20:31:20Z" level=info msg="unchanged Application" app=dev01-neuron-trino appSet=community-dev01-app-catalog
time="2022-02-01T20:31:20Z" level=info msg="end reconcile" requeueAfter=3m0s

We didn't find any timeout, caching or tuning parameters in the docs. Also I am not sure, if the caching settings of ArgoCD Application Controller or RepoServer could play a role here.

Thanks for your help!

dan1el-k avatar Feb 01 '22 21:02 dan1el-k

I further analysed the behaviour and found actually the issues, causing the crazy git polling intervals in our case.

Issue 1 - Application Syncs of Application Controller (with app-resync schedule) triggers ApplicationSet

The Application controller app-resync (by default every 3m) triggers ! in addition the ApplicationSet controller. In my opinion, this contradicts the desired behaviour of the reconciliation schedule built into the ApplicationSet.

Issue 2 - AppSync trigger acts for whole AppSet instead of on per/app basis

All generators of an AppSet get triggered, even though just 1 application was generated out of 1 generator. This results in, that obviously generators runs which have no application defined.

So: AppSet with generators found 0 applications takes reconcile schedule AppSet with generators found 1 or more applications runs multiple times at reconcile schedule + appsync schedule

Issue 3 - incorrect detected applications, by using multiple generators in 1 AppSet

In the log applications are considered as generated ("generated 1 applications") event though the condition of the generator wasnt valid. So it looks like an application was generated, but of course there wasnt an ArgoCD Application object created. This impacts also Issue 2 in terms of sync and reconcilation behaviour.

Test & Validation:

We have defined 2 Git Generators,

  • 1 listening on environments/dev02/kustomize/*.yaml
  • 1 listening on environments/dev02/helm/*.yaml There is only defined 1 application config in the `environments/dev02/kustomize/cortex-template-kustomize.yaml

Params set:

  • in application-controller: --app-resync=1m
  • default reconcilation schedule =3m

SyncPolicy of template in ApplicationSets

spec: 
  template: 
    spec:
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        retry:
          backoff:
            duration: 5s
            factor: 2
            maxDuration: 2m
          limit: 3
        syncOptions:
          - CreateNamespace=true
          - ApplyOutOfSyncOnly=true

**Generators in separate ApplicationSets - 1 for kustomize folder 1 for helm folder **

I0210 09:08:04.561322       1 request.go:665] Waited for 1.012000544s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/cert-manager.io/v1beta1?timeout=32s
time="2022-02-10T09:08:07Z" level=info msg="Starting configmap/secret informers"
time="2022-02-10T09:08:07Z" level=info msg="Configmap/secret informer synced"
time="2022-02-10T09:08:07Z" level=info msg="processing event for cluster secret" name=argocd-cortex-cluster-config namespace=cortex-argocd type=createSecretEventHandler
time="2022-02-10T09:08:07Z" level=info msg="listed ApplicationSets" count=2 type=createSecretEventHandler
time="2022-02-10T09:08:07Z" level=info msg="Initializing https://my.url/myorg/cortex-bootstrap-app-catalog.git to /tmp/https___my.url_myorg_cortex-bootstrap-app-catalog"
time="2022-02-10T09:08:07Z" level=info msg="rm -rf /tmp/https___my.url_myorg_cortex-bootstrap-app-catalog" dir= execID=a0188
time="2022-02-10T09:08:07Z" level=info msg=Trace args="[rm -rf /tmp/https___my.url_myorg_cortex-bootstrap-app-catalog]" dir= operation_name="exec rm" time_ms=4.382143
time="2022-02-10T09:08:07Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=96365
time="2022-02-10T09:08:08Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=806.503682
time="2022-02-10T09:08:08Z" level=info msg="git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=dbd07
time="2022-02-10T09:08:08Z" level=info msg=Trace args="[git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=9.942886999999999
time="2022-02-10T09:08:08Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=7558f
time="2022-02-10T09:08:08Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.68829
time="2022-02-10T09:08:08Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=ad507
time="2022-02-10T09:08:08Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.892303
time="2022-02-10T09:08:08Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc0001669a0 <nil>}"
time="2022-02-10T09:08:08Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog-kustomize
time="2022-02-10T09:08:08Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:08:08Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=d6550
time="2022-02-10T09:08:09Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=663.393151
time="2022-02-10T09:08:09Z" level=info msg="git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=8d834
time="2022-02-10T09:08:09Z" level=info msg=Trace args="[git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=10.009713
time="2022-02-10T09:08:09Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=82125
time="2022-02-10T09:08:09Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.6795970000000002
time="2022-02-10T09:08:09Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=4399d
time="2022-02-10T09:08:09Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.942267
time="2022-02-10T09:08:09Z" level=info msg="generated 0 applications" generator="{<nil> <nil> 0xc0012149c0 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:08:09Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:09:54Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=63ba2
time="2022-02-10T09:09:55Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=622.090728
time="2022-02-10T09:09:55Z" level=info msg="git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=fcd7e
time="2022-02-10T09:09:55Z" level=info msg=Trace args="[git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=16.999429
time="2022-02-10T09:09:55Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=a5cbb
time="2022-02-10T09:09:55Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=14.517002
time="2022-02-10T09:09:55Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=27610
time="2022-02-10T09:09:55Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.778172
time="2022-02-10T09:09:55Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000644000 <nil>}"
time="2022-02-10T09:09:55Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog-kustomize
time="2022-02-10T09:09:55Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:10:54Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=6d2dc
time="2022-02-10T09:10:55Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=605.9635420000001
time="2022-02-10T09:10:55Z" level=info msg="git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=b2511
time="2022-02-10T09:10:55Z" level=info msg=Trace args="[git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=18.200893999999998
time="2022-02-10T09:10:55Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=fa85e
time="2022-02-10T09:10:55Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=6.370879
time="2022-02-10T09:10:55Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=8efe8
time="2022-02-10T09:10:55Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=4.789059
time="2022-02-10T09:10:55Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc0001669a0 <nil>}"
time="2022-02-10T09:10:55Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog-kustomize
time="2022-02-10T09:10:55Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:11:08Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=7a3e2
time="2022-02-10T09:11:09Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=808.244713
time="2022-02-10T09:11:09Z" level=info msg="git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=d462a
time="2022-02-10T09:11:09Z" level=info msg=Trace args="[git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.748011
time="2022-02-10T09:11:09Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=7c801
time="2022-02-10T09:11:09Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.637234
time="2022-02-10T09:11:09Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=e5595
time="2022-02-10T09:11:09Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.153054
time="2022-02-10T09:11:09Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000644b00 <nil>}"
time="2022-02-10T09:11:09Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog-kustomize
time="2022-02-10T09:11:09Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:11:09Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=e15de
time="2022-02-10T09:11:10Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=547.3157789999999
time="2022-02-10T09:11:10Z" level=info msg="git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=39ba8
time="2022-02-10T09:11:10Z" level=info msg=Trace args="[git checkout --force 2e5523e12c9185726d88b55f4954643f3695d50e]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=17.391203
time="2022-02-10T09:11:10Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=2cf25
time="2022-02-10T09:11:10Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.825793
time="2022-02-10T09:11:10Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=88b06
time="2022-02-10T09:11:10Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.817794
time="2022-02-10T09:11:10Z" level=info msg="generated 0 applications" generator="{<nil> <nil> 0xc001214340 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:11:10Z" level=info msg="end reconcile" requeueAfter=3m0s

Both generators in 1 ApplicationSets:

I0210 09:40:27.766949       1 request.go:665] Waited for 1.039237259s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/apiserver.openshift.io/v1?timeout=32s
time="2022-02-10T09:40:30Z" level=info msg="Starting configmap/secret informers"
time="2022-02-10T09:40:30Z" level=info msg="Configmap/secret informer synced"
time="2022-02-10T09:40:30Z" level=info msg="processing event for cluster secret" name=argocd-cortex-cluster-config namespace=cortex-argocd type=createSecretEventHandler
time="2022-02-10T09:40:30Z" level=info msg="listed ApplicationSets" count=1 type=createSecretEventHandler
time="2022-02-10T09:40:30Z" level=info msg="Initializing https://my.url/myorg/cortex-bootstrap-app-catalog.git to /tmp/https___my.url_myorg_cortex-bootstrap-app-catalog"
time="2022-02-10T09:40:30Z" level=info msg="rm -rf /tmp/https___my.url_myorg_cortex-bootstrap-app-catalog" dir= execID=4ba54
time="2022-02-10T09:40:30Z" level=info msg=Trace args="[rm -rf /tmp/https___my.url_myorg_cortex-bootstrap-app-catalog]" dir= operation_name="exec rm" time_ms=2.9204220000000003
time="2022-02-10T09:40:30Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=d0584
time="2022-02-10T09:40:31Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1050.523078
time="2022-02-10T09:40:31Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=c7103
time="2022-02-10T09:40:31Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=7.990499
time="2022-02-10T09:40:31Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=b0a26
time="2022-02-10T09:40:31Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.23286
time="2022-02-10T09:40:31Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=135b1
time="2022-02-10T09:40:31Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.708669
time="2022-02-10T09:40:31Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000796b00 <nil>}"
time="2022-02-10T09:40:31Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=8a9fe
time="2022-02-10T09:40:32Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=912.952977
time="2022-02-10T09:40:33Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=e483b
time="2022-02-10T09:40:33Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=7.752903
time="2022-02-10T09:40:33Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=26b4b
time="2022-02-10T09:40:33Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.847268
time="2022-02-10T09:40:33Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=906f7
time="2022-02-10T09:40:33Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.8119669999999997
time="2022-02-10T09:40:33Z" level=info msg="generated 1 applications" generator="{<nil> <nil> 0xc0001f9520 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:40:33Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog
time="2022-02-10T09:40:33Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:40:33Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=08ed5
time="2022-02-10T09:40:33Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=623.422377
time="2022-02-10T09:40:34Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=fd05b
time="2022-02-10T09:40:34Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=6.243355
time="2022-02-10T09:40:34Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=124df
time="2022-02-10T09:40:34Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=14.984241
time="2022-02-10T09:40:34Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=2300a
time="2022-02-10T09:40:34Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.246395
time="2022-02-10T09:40:34Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc0006e9a20 <nil>}"
time="2022-02-10T09:40:34Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=c4627
time="2022-02-10T09:40:34Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=648.9491370000001
time="2022-02-10T09:40:35Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=2bf00
time="2022-02-10T09:40:35Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=8.049901
time="2022-02-10T09:40:35Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=48b4a
time="2022-02-10T09:40:35Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=17.714884
time="2022-02-10T09:40:35Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=443e5
time="2022-02-10T09:40:35Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.73193
time="2022-02-10T09:40:35Z" level=info msg="generated 1 applications" generator="{<nil> <nil> 0xc0010944e0 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:40:35Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog
time="2022-02-10T09:40:35Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:40:54Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=58c02
time="2022-02-10T09:40:55Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=769.7546520000001
time="2022-02-10T09:40:55Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=dc441
time="2022-02-10T09:40:55Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=25.166819999999998
time="2022-02-10T09:40:55Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=f407d
time="2022-02-10T09:40:55Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=5.453339
time="2022-02-10T09:40:55Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=eacdd
time="2022-02-10T09:40:55Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=6.432641
time="2022-02-10T09:40:55Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc00091c9a0 <nil>}"
time="2022-02-10T09:40:55Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=21485
time="2022-02-10T09:40:56Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=651.722648
time="2022-02-10T09:40:56Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=905fd
time="2022-02-10T09:40:56Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=21.887553
time="2022-02-10T09:40:56Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=bf221
time="2022-02-10T09:40:56Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.657407
time="2022-02-10T09:40:56Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=44cb6
time="2022-02-10T09:40:56Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=4.484077
time="2022-02-10T09:40:56Z" level=info msg="generated 1 applications" generator="{<nil> <nil> 0xc00091a1a0 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:40:56Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog
time="2022-02-10T09:40:56Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:41:54Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=004b8
time="2022-02-10T09:41:55Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=774.933203
time="2022-02-10T09:41:55Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=20756
time="2022-02-10T09:41:55Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=23.465747
time="2022-02-10T09:41:55Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=e2dbc
time="2022-02-10T09:41:55Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=13.889384
time="2022-02-10T09:41:55Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=7a863
time="2022-02-10T09:41:55Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=10.664876999999999
time="2022-02-10T09:41:55Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000bcfa20 <nil>}"
time="2022-02-10T09:41:55Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=f883d
time="2022-02-10T09:41:56Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=716.621041
time="2022-02-10T09:41:56Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=3860f
time="2022-02-10T09:41:56Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=7.307349
time="2022-02-10T09:41:56Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=1dfde
time="2022-02-10T09:41:56Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.31895
time="2022-02-10T09:41:56Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=54a4a
time="2022-02-10T09:41:56Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1.7536120000000002
time="2022-02-10T09:41:56Z" level=info msg="generated 1 applications" generator="{<nil> <nil> 0xc0010ee9c0 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:41:56Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog
time="2022-02-10T09:41:56Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:42:54Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=85551
time="2022-02-10T09:42:55Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=857.911354
time="2022-02-10T09:42:56Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=afdf9
time="2022-02-10T09:42:56Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=24.217273000000002
time="2022-02-10T09:42:56Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=b3a2b
time="2022-02-10T09:42:56Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=15.759358
time="2022-02-10T09:42:56Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=92f32
time="2022-02-10T09:42:56Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=7.835283
time="2022-02-10T09:42:56Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc0006e8c60 <nil>}"
time="2022-02-10T09:42:56Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=80650
time="2022-02-10T09:42:56Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=719.8080759999999
time="2022-02-10T09:42:57Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=d0ef9
time="2022-02-10T09:42:57Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=7.7646310000000005
time="2022-02-10T09:42:57Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=83b20
time="2022-02-10T09:42:57Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=35.545219
time="2022-02-10T09:42:57Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=ffdae
time="2022-02-10T09:42:57Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=22.046021999999997
time="2022-02-10T09:42:57Z" level=info msg="generated 1 applications" generator="{<nil> <nil> 0xc001094340 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:42:57Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog
time="2022-02-10T09:42:57Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:43:33Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=02b58
time="2022-02-10T09:43:33Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=679.4918210000001
time="2022-02-10T09:43:34Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=abcc5
time="2022-02-10T09:43:34Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=4.769028
time="2022-02-10T09:43:34Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=89924
time="2022-02-10T09:43:34Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=17.757107
time="2022-02-10T09:43:34Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=9c6df
time="2022-02-10T09:43:34Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=8.375619
time="2022-02-10T09:43:34Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc0000a8420 <nil>}"
time="2022-02-10T09:43:34Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=b55e2
time="2022-02-10T09:43:34Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=713.510001
time="2022-02-10T09:43:35Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=ccda4
time="2022-02-10T09:43:35Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=7.259663000000001
time="2022-02-10T09:43:35Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=23de9
time="2022-02-10T09:43:35Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.278408
time="2022-02-10T09:43:35Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=30593
time="2022-02-10T09:43:35Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=9.777404
time="2022-02-10T09:43:35Z" level=info msg="generated 1 applications" generator="{<nil> <nil> 0xc001094680 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:43:35Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog
time="2022-02-10T09:43:35Z" level=info msg="end reconcile" requeueAfter=3m0s
time="2022-02-10T09:43:54Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=7ccd9
time="2022-02-10T09:43:55Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=1044.056328
time="2022-02-10T09:43:55Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=9dc61
time="2022-02-10T09:43:55Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=18.536571000000002
time="2022-02-10T09:43:55Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=d683b
time="2022-02-10T09:43:55Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=27.240259
time="2022-02-10T09:43:55Z" level=info msg="git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=7e67b
time="2022-02-10T09:43:55Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/kustomize/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=3.5885700000000003
time="2022-02-10T09:43:55Z" level=info msg="generated 1 applications" generator="{<nil> <nil> <nil> <nil> <nil> <nil> 0xc000184f20 <nil>}"
time="2022-02-10T09:43:55Z" level=info msg="git fetch origin cortex-test/cortex-test01 --tags --force" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=4bb27
time="2022-02-10T09:43:56Z" level=info msg=Trace args="[git fetch origin cortex-test/cortex-test01 --tags --force]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=626.15694
time="2022-02-10T09:43:56Z" level=info msg="git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=7a924
time="2022-02-10T09:43:56Z" level=info msg=Trace args="[git checkout --force 3421e48751e627408cc9bcc491fcd89299bde983]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=29.760798
time="2022-02-10T09:43:56Z" level=info msg="git clean -fdx" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=2bfc7
time="2022-02-10T09:43:56Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=2.501184
time="2022-02-10T09:43:56Z" level=info msg="git ls-files --full-name -z -- environments/dev02/helm/*.yaml" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog execID=1a6d4
time="2022-02-10T09:43:56Z" level=info msg=Trace args="[git ls-files --full-name -z -- environments/dev02/helm/*.yaml]" dir=/tmp/https___my.url_myorg_cortex-bootstrap-app-catalog operation_name="exec git" time_ms=22.200125
time="2022-02-10T09:43:56Z" level=info msg="generated 1 applications" generator="{<nil> <nil> 0xc000f921a0 <nil> <nil> <nil> <nil> <nil>}"
time="2022-02-10T09:43:56Z" level=info msg="unchanged Application" app=dev02-cortex-template-kustomize appSet=dev02-app-catalog
time="2022-02-10T09:43:56Z" level=info msg="end reconcile" requeueAfter=3m0s

dan1el-k avatar Feb 10 '22 10:02 dan1el-k