Allow use of existing namespace

[ba-work]

I want to be able use an existing namespace without the flux_bootstrap_git taking it over.

Currently I can use an existing secret and an existing namespace (obviously, the secret has to be somewhere), but then the bootstrap resource takes over the namespace and on destroys (from taints/changes/destroys/etc), it wipes out the secret (since its in the namespace).

This behavior also makes the bootstrap resource incompatible with lifecycle { create_before_destroy = true } since the destroy afterwards also wipes out the entire namespace.

I think something like existing_namespace = true would be sufficient.

It could also be argued that this behavior should happen in cases of disable_secret_creation = true since the secret must exist in the same namespace.

example tf that i'm using:

resource "tls_private_key" "flux" { algorithm = "RSA" }

resource "kubernetes_namespace" "flux" {
  metadata { name = "flux-system" }
  lifecycle { ignore_changes = [metadata[0].labels] } # have to ignore labels that flux adds
}

resource "kubernetes_secret" "flux" {
  metadata {
    name      = "flux-system"
    namespace = kubernetes_namespace.flux.metadata[0].name
  }
  data = {
    "identity"     = tls_private_key.flux.private_key_pem
    "identity.pub" = tls_private_key.flux.public_key_openssh
    "known_hosts"  = var.repository.ssh_host_key
  }
}

resource "flux_bootstrap_git" "bootstrap" {
  path                    = "flux/clusters/${var.cluster_name}"
  disable_secret_creation = true
  
  # i want this
  # existing_namespace = true
}

[ba-work]

Current workaround for me is to carefully tf state rm flux_bootstrap_git.bootstrap when appropriate, then tf apply.

[swade1987]

Hello @ba-work 👋

I hope all is well with you. I'm just contacting you as part of our initiative to ensure the quality and responsiveness of our project's issue tracking. Thank you for your previous contributions by reporting issues you've encountered; your engagement is invaluable to the health and progress of our project.

I've reviewed the issue you reported and noted that it involves using supported resources, which is great. However, I also noticed that there has been little activity on this issue lately. We understand that circumstances change and priorities shift, which may affect the urgency or relevance of an issue.

We're implementing a policy regarding inactive issues to keep our project's issue tracker effective and up-to-date. If there's no further activity on this issue within the next 3 weeks, we will mark it inactive and close it. This approach helps us maintain focus on current and actively pursued concerns, ensuring resources are well allocated.

That said, we are still very much interested in resolving the issue you've encountered. If the problem persists or if you have any new information that could help us address it, please don't hesitate to update the issue or comment. Your input could be crucial in finding a resolution. Alternatively, if the issue no longer affects you or has been resolved, let us know so we can close it with the right resolution.

We're here to support and work with you to overcome any challenges you face using our project. Your continued feedback and participation are essential to our community's success.

Thank you once again for your contribution. I look forward to your update and am here to assist with any further questions or issues you might have.

Best regards,

Steve

---

Note: I would highly recommend letting flux_bootstrap_git to use the flux-system. You may find this repository useful to you. It's an example of how I have managed the bootstrapping of clusters previously by dropping any additional resources into a flux-bootstrap cluster which flux then reconciles.

[ba-work]

That is what I'm doing.

The example I give is using the flux-system namespace, the root of the problem is that flux_bootstrap_git takes ownership of the namespace (ie. a tf destroy deletes the namespace) which logically conflicts with the option of disable_secret_creation = true since the secret must exist in that very same namespace.

I'm just asking for way to tell the provider not to own the namespace, similar to how it has an option not to own the secret.

[swade1987]

@ba-work

In my opinion (and how I have set it up in the past), I let flux_bootstrap_git manage the flux-system namespace and deploy the flux pods and the secret it requires to speak to your git repository.

Then, in the git repository, I deploy the remainder of my resources into a new namespace called flux-bootstrap.

I have a full example of this available here, and I think it may help you.

The sections that will be of interest to you are:

• https://github.com/swade1987/flux2-multi-cluster-bootstrap-repo/blob/main/terraform/main.tf#L65-L90
• https://github.com/swade1987/flux2-multi-cluster-bootstrap-repo/tree/main/kustomize/_base/gitrepositories

[ba-work]

That's fine, but right now the provider allows you pre create a secret in the namespace, my example in the report is the smallest possible using tf. In a more productionized env (and indeed in my real uses case) its more complex and there are more "things" in the ns; having this resource secretly take over the ns (and by extension the pre created secret) is not good!

Regardless of how you've done it in the past, this is a bug/issue. Perhaps instead we should be discussing the removal of disable_secret_creation?

[swade1987]

@ba-work i don't necessarily agree with this being a bug. You should keep flux-system clean and just let it bootstrap flux. Then put all other resources in other namespaces like I have in the repository example. This stops any issues and allows flux_bootstrap_git to manage the flux-system namespace via terraform without issues.

This is the cleanest implementation and one I have used many times at clients of mine without issue.

[ba-work]

If you are that adamant, then I guess the only question I have is: What is the appropriate way to use disable_secret_creation = true? I must be misunderstanding its purpose since if I use it in any capacity, doing following results in an error:

tf apply -auto-approve && tf destroy -auto-approve && tf apply -auto-approve;

(the second apply won't work since the destroy will have deleted the secret that it shouldn't be touching.)

[swade1987]

@ba-work I use that to manage the lifecycle of the SSH keypair that flux uses to talk to your initial git repository in Terraform rather than let flux manage it. If you need to rotate the key then its best to use this, I actually have an example of how to use it here

[ba-work]

... that linked example looks exactly like the example I wrote in the initial report. I feel like you haven't extensively used your own example otherwise you would eventually encounter the behaviour I'm talking about. I only opened the issue because this resource deleted the namespace and secret in an environment I have with basically the same syntax.

Feel free to close issue, we are going in circles and I've migrated to just installing flux with a null_resource and bash script for almost a year at this point.