Ability to Obtain Token/Credentials for Helm Provider?

[steve-gray]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

When creating an OKE cluster like this, we usually want to provision a series of baseline services into the cluster. Our standard way of doing this today is Helm - however it appears that there's no output on this module that can be used to feed into a Helm step. Challenge areas are:

• Deploying kubernetes_manifests into the cluster after provisioning
• Running helm packages into the cluster after provisioning

We've looked at what it'd take to do this all via null_resources and SSHing via the bastion, but its super hard yards and means 99% of the value of terraform is lost for those resource types.

[12345ieee]

The way me and my team is doing this is to configure the helm provider like this:

provider "helm" {
  kubernetes {
    config_path = "generated/kubeconfig"
  }
}

and then add a depends_on = [module.oke] to the helm_release resources.

This is indeed not ideal, as there is no real link between the oke module and the helm resources and the kubeconfig file regeneration is a bit fragile, but it works.

It helps us that we just want to deploy ArgoCD this way, then we have it bootstrap the whole cluster content, but I imagine if you want to handle every deployed app in terraform via helm_release it gets annoying.

[steve-gray]

Ah, so you're relying on the path being relative to the parent module? Interesting. I'll give that a shot. Are you using TF cloud for this @12345ieee?

[hyder]

A couple of things:

1. We use kubeconfig v2 so that requests to the API server are authenticated. This authentication is done via oci cli. We've configured this on the operator host. So, if you've enabled the bastion and operator hosts and instance_principal for operator, then you can use null_resource via TF. I wrote about using TF and helm once but I'm not sure I particularly TF managing helm + infra in 1 big bucket.
2. We've removed the local generated/kubeconfig now recently. kubeconfig were overwriting each other when used in a multiple cluster provisioning scenario. If you're still using/needing this, please open a new ticket. Otherwise, I suggest running this from the operator.

@devoncrouse FYI re: kubeconfig

[12345ieee]

@hyder We are on v4.2.5 because v4.2.6 has the blocker detailed in #591 that stops me from upgrading.

If newer version did away with any local kubeconfig output, the future workflow would be using a null resource to manually run helm install ... on the bastion instead of using the helm provider?

I agree that managing the cluster content from tf isn't THE WAY (TM), but it's important for us to deploy a chart that bootstraps the cluster, and to do so via the helm provider and a safe helm_release resource.

[12345ieee]

So @steve-gray I was not aware of this transition, the workflow I described above works for sure up to v4.2.6, sorry for the confusion.

[hyder]

I'm closing this, following the discussion in #601.