Disable `kustomization_override` in `flux_bootstrap_git` resource

[aditditto]

Hi, thanks a lot for the terraform provider.

I would like to ask if it is possible to disable the kustomization_override in flux_bootstrap_git resource, so that the resource doesn't check for any changes in the Kustomization file.

My use case is using the flux_bootstrap_git only for bootstrapping, and modifying the kustomization.yaml file through git repo in the flux-system directory instead of in terraform.

[phillebaba]

These types of things become very difficult to implement. I am already stretching the possibilities of Terraforms state diff by a bit as it is. Adding exceptions would make things more challenging.

Could I ask what your use case for this would be? Why not manage everything with Terraform?

[aditditto]

My use case is to use the flux_bootstrap_git strictly for bootstrapping, and to do any YAML modification in the bootstrapped git repo. There is just an aversion in my organization to install YAML through terraform, but if it is not possible then overriding the Kustomization in terraform would work just fine

[stefanprodan]

@aditditto the kustomization_override should be used only for Flux and since you decided to bootstrap with Terraform instead of the Flux CLI, the Flux upgrades must be managed from TF. If you patch the Flux controllers outside of TF, then how are you going to upgrade?

What type of changes are you making to the Kustomization file? How often do you make them?

[aditditto]

@aditditto the kustomization_override should be used only for Flux and since you decided to bootstrap with Terraform instead of the Flux CLI, the Flux upgrades must be managed from TF. If you patch the Flux controllers outside of TF, then how are you going to upgrade?

What type of changes are you making to the Kustomization file? How often do you make them?

I have never tried upgrading, but my best guess for how to upgrade Flux would be to change the version field in the flux_bootsrap_git resource.

Currently we are still migrating to Kubernetes, so I am still testing out Flux. Currently I only want to patch the service account to use GKE Workload Identity, so something like this

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - gotk-components.yaml
  - gotk-sync.yaml
patches:
  - patch: |
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: source-controller
        annotations:
          iam.gke.io/gcp-service-account: <identity-name>      
    target:
      kind: ServiceAccount
      name: source-controller

[Sierra1011]

I've got a similar issue; bootstrapping a new cluster and then letting Flux self-manage from the repo. Problem is that the kustomization.yaml being bootstrapped is blank, and doesn't contain the service account details, so it overwrites it each time and, besides causing a redeployment, it makes commits and gets involved with our CI :upside_down_face:

I've tried lifecycle { ignore_changes = all } but it still tries to update it every time. Adding the kustomization_override file to the Terraform directory was the only way to prevent it re-bootstrapping every time.

[Shaked]

Seems like a dup https://github.com/fluxcd/terraform-provider-flux/issues/530. Is there a plan to support this?

[swade1987]

Echoing @stefanprodan's insights, it's crucial to handle Flux's configuration exclusively through Terraform, especially when utilizing flux_bootstrap_git. As mentioned above, this approach ensures smooth operations and helps avoid issues that might arise otherwise. I will close this issue as @stefanprodan stated the fix above.