Changes on GitRepository ignored after changing ref type

[Trik16]

trafficstars

Describe the bug

Due to our release pipeline I noticed that I have troubles to update the gitrepository if I change the ref type from tag to branch or vice versa. Even if I reconcile the source the changes is ignored and I had to delete manually the sources and load from scratch with flux-system kustomize reconciliation.

Steps to reproduce

1. After bootstraping the cluster create 2 simple resources a kustomization and a gitrepositories on your bootstrap repository (in my case is the same in which I keep the kustomization files but different folders)

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  name: deployment-stable
  namespace: flux-system
spec:
  interval: 30s
  ref:
    tag: your-tag
  url: ssh://[email protected]/your-organization/your-repo
  
  ---
 apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: deployment-stable
  namespace: flux-system
spec:
  interval: 5m0s
  path: ./kustomize/overlays/stable
  prune: true
  sourceRef:
    kind: GitRepository
    name: deployment-stable
  targetNamespace: stable
  

2. Wait for reconciliation, verify the correct creation of the deployment (./kustomize/overlays/stable) on the chosen namespace and then change from tag to branch in the ref field:

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  name: deployment-stable
  namespace: flux-system
spec:
  interval: 30s
  ref:
    branch: main
  url: ssh://[email protected]/your-organization/your-repo
  

3. Wait for automatic reconciliation or do it manually. The changes is not applied.
4. Delete the resource manually from the cluster and then reconcile the flux-system kustomization

Expected behavior

I want that flux applies automatically the changes from the ref field specified even if it changes type

Screenshots and recordings

No response

OS / Distro

Ubuntu 24.04 LTS

Flux version

flux: v2.2.3

Flux check

► checking prerequisites ✗ flux 2.2.3 <2.3.0 (new CLI version is available, please upgrade) ✔ Kubernetes 1.26.7 >=1.26.0-0 ► checking version in cluster ✔ distribution: flux-v2.2.3 ✔ bootstrapped: true ► checking controllers ✔ helm-controller: deployment ready ► ghcr.io/fluxcd/helm-controller:v0.37.4 ✔ image-automation-controller: deployment ready ► ghcr.io/fluxcd/image-automation-controller:v0.37.1 ✔ image-reflector-controller: deployment ready ► ghcr.io/fluxcd/image-reflector-controller:v0.31.2 ✔ kustomize-controller: deployment ready ► ghcr.io/fluxcd/kustomize-controller:v1.2.2 ✔ notification-controller: deployment ready ► ghcr.io/fluxcd/notification-controller:v1.2.4 ✔ source-controller: deployment ready ► ghcr.io/fluxcd/source-controller:v1.2.4 ► checking crds ✔ alerts.notification.toolkit.fluxcd.io/v1beta3 ✔ buckets.source.toolkit.fluxcd.io/v1beta2 ✔ gitrepositories.source.toolkit.fluxcd.io/v1 ✔ helmcharts.source.toolkit.fluxcd.io/v1beta2 ✔ helmreleases.helm.toolkit.fluxcd.io/v2beta2 ✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2 ✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2 ✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2 ✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta1 ✔ kustomizations.kustomize.toolkit.fluxcd.io/v1 ✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2 ✔ providers.notification.toolkit.fluxcd.io/v1beta3 ✔ receivers.notification.toolkit.fluxcd.io/v1 ✔ all checks passed

Git provider

GitHub

Container Registry provider

No response

Additional context

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[stefanprodan]

I suspect this is because the tag field is still in etcd and when both branch and tag are specified, the tag gets priority.

Can you please post kubectl get gitrepo -o yaml —show-managed-fields?

PS You should really update the API version for both GitRepo and Kustomization to v1. The beta versions you’re using have been deprecated long time ago.

[Trik16]

Can you please post kubectl get gitrepo -o yaml —show-managed-fields?

apiVersion: v1
items: []
kind: List
metadata:
  resourceVersion: ""

[stefanprodan]

@Trik16 run that command for a GitRepo which has the reported issue

kubectl -n flux-system get gitrepo  deployment-stable -o yaml --show-managed-fields

[Trik16]

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  annotations:
    reconcile.fluxcd.io/requestedAt: "2024-06-13T18:03:21.682398+02:00"
  creationTimestamp: "2024-06-13T15:50:40Z"
  finalizers:
  - finalizers.fluxcd.io
  generation: 4
  labels:
    kustomize.toolkit.fluxcd.io/name: flux-system
    kustomize.toolkit.fluxcd.io/namespace: flux-system
  name: deployment-stable
  namespace: flux-system
  resourceVersion: "109522355"
  uid: 2da3732c-5aa6-4917-b9a5-c05a049c68ec
spec:
  include:
  - fromPath: bopen/stable
    repository:
      name: configuration-stable
    toPath: kustomize/overlays/stable
  interval: 30s
  ref:
    branch: main
  secretRef:
    name: flux-system
  timeout: 60s
  url: ssh://[email protected]/org/repo
status:
  artifact:
    digest: sha256:f98efed090d584cb496c48b46f9e909ded9d110eb70b1ee2046dd834d88be09e
    lastUpdateTime: "2024-06-13T16:32:02Z"
    path: gitrepository/flux-system/deployment-stable/f30ea408d2e71cf29e022f04382b75e95643a346.tar.gz
    revision: main@sha1:f30ea408d2e71cf29e022f04382b75e95643a346
    size: 750263
    url: http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/deployment-stable/f30ea408d2e71cf29e022f04382b75e95643a346.tar.gz
  conditions:
  - lastTransitionTime: "2024-06-13T16:32:02Z"
    message: stored artifact for revision 'main@sha1:f30ea408d2e71cf29e022f04382b75e95643a346'
    observedGeneration: 4
    reason: Succeeded
    status: "True"
    type: Ready
  - lastTransitionTime: "2024-06-13T15:51:14Z"
    message: stored artifact for revision 'main@sha1:f30ea408d2e71cf29e022f04382b75e95643a346'
    observedGeneration: 4
    reason: Succeeded
    status: "True"
    type: ArtifactInStorage
  includedArtifacts:
  - digest: sha256:703d923fe581a88e81f80f5cf535dbf993ad685cc66261be1cf8adebccbe022a
    lastUpdateTime: "2024-06-13T16:31:36Z"
    path: gitrepository/flux-system/configuration-stable/1257f66fcecac8daeb7928fdb00343703a68a68a.tar.gz
    revision: main@sha1:1257f66fcecac8daeb7928fdb00343703a68a68a
    size: 359119
    url: http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/configuration-stable/1257f66fcecac8daeb7928fdb00343703a68a68a.tar.gz
  lastHandledReconcileAt: "2024-06-13T18:03:21.682398+02:00"
  observedGeneration: 4
  observedInclude:
  - fromPath: bopen/stable
    repository:
      name: configuration-stable
    toPath: kustomize/overlays/stable

[stefanprodan]

You missed --show-managed-fields

[Trik16]

I did not, this is what I get

[stefanprodan]

Ok I suspect this issue is due to having deprecated API versions in Git.

To fix the APIs and also to make sure changing from tag to branch is always considered by Flux, you need to use ref.name https://fluxcd.io/flux/components/source/gitrepositories/#name-example.

When setting a branch do:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: <repository-name>
spec:
  ref:
    name: "refs/heads/main"

When setting a tag do:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: <repository-name>
spec:
  ref:
    name: "refs/tags/v1.0.0"

[Trik16]

ok, thanks

[aservedio]

I noticed this issue a few times on latest flux 2.5.1:

• I start off with a GitRepository and a spec/ref/tag
• I switch to a spec/ref/branch and comment out tag
• I end up with flux still reconciling on tag for flux kustomizations using it

Tried a few more things after finding this thread:

• The kubectl command above shows both ref/branch and ref/tag are defined even though my yaml has only branch.
• Using ref/name instead I see flux reconciling to "refs/heads/my-branch" (nice!)
• In the server spec I now see both ref/name and my old ref/tag still there