support core, read-only git repos in flux_bootstrap terraform resource

[kingdonb]

@phillebaba Have you seen this? I think we had some related discussion at the Dev Meeting, last week or this week. Jeff showed up at Bug Scrub this week with questions, and was very complimentary about the new bootstrap provider 🎉

Discussed in https://github.com/fluxcd/flux2/discussions/3580

^{Originally posted by 0x6a77 February 12, 2023} in the documents-based terraform datas we can share a core git repo across multiple clusters and that repo is read-only to flux, but managed centrally. under flux_bootstrap this core repo now gets git push from every cluster which has the surprising consequence of deleting every cluster when you delete one. yoink!

a core repo is a nice model that we've used for years: the teams get their own, custom repo for their workloads, but otherwise the central services get managed from the core repo.

it feels like flux_boostrap resource could support this with a small change: mark the resource "git read-only." on apply the provider would skip any logic to update/push its gotk/kustomization. in read-only mode it might be nice to warn on diffs and fail on missing files. on destroy it would just skip anything related to file delete/push.

the team that manages the core repo would use it in read/write mode and take responsibility for making sure resources match versions for all teams that share it. (which over the years has been reasonably stable, so this would not be a lot of work.)

[kingdonb]

In the linked discussion thread, there is also a proof of concept implementation. The issue is a repo admin key is needed at bootstrap time, but the read-only deploy key is typically going to be the preferred one after that. (I'm sure we did have this discussion already, and linked to the YouTube channel, but I'm not sure when.)

[moritzschmitz-oviva]

Perhaps providing an install resource is an option as well?

[swade1987]

@kingdonb would this resolve the issue using https://github.com/fluxcd/terraform-provider-flux/pull/614 when its merged and released? Having a way to run flux_bootstrap_git without actually writing the files in Git seems weird it feels like its not really performing a bootstrap (unless I am missing something) 🤔

[kingdonb]

I'm really not sure, we should ask @0x6a77 (jeff who originally posted the thread?)

I don't really have context for the use case described. I am not using tf-provider-flux with teams or managing GitOps for teams in a way where I would have direct experience of what problem is being solved.

[stefanprodan]

Not pushing to Git is out of scope for this provider. The only way to install Flux without bootstrap is by using the Helm charts https://github.com/fluxcd/terraform-provider-flux/tree/main/examples/helm-install