Suport for Nested Helm/Kubernetes Providers on Terraform Cloud

[steve-gray]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

At present it's not possible to use OCI end to end for Kubernetes on Terraform cloud - you can build the cluster but not practically put any workload into it - as the only auth method supported by generation from the terraform OCI provider gives a kubeconfig that requires the oci CLI tools. These aren't present and are generally impractical to install on Terraform cloud setups or other CI-based ways to run terraform scripts, potentially taking many times the install time and introducing a lot of jank/reliability problems (curling and unpacking/installing python in an ephemeral container etc).

The ideal change here is a new resource that generates some sort of client certificate/key pair that can be used as input to the "kubernetes" provider https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#credentials-config

With AWS/GCP/Azure this is possible, it'd be great to get OCI to the same level of automation-friendliness.

[YoshiChladny]

Running into the same issue.

This is a huge automation blocker, especially when using terraform to bootstrap flux in your cluster.

[steve-gray]

@ravinitp how is this complete? Has the ability been added somewhere?

[ravinitp]

We are very sorry that we couldn't respond to each and every issue reported on GitHub. Although we have refined the process to prioritize customer issues on GitHub, since this issue was reported a while ago, there is a good chance it may have been fixed in the latest version of Terraform Provider OCI.

If you are still experiencing this issue, please create a new issue and label it as Bug.

[ravinitp]

affected_resources=oci_containerengine_cluster

[tf-oci-pub]

Thank you for reporting the issue. We observed the affected resources are not provided in the description or it's incorrect. We request you to add it in issue description as mentioned in below format. Example: affected_resources = oci_core_instance , oci_core_instances

If it's not related to any particular resource then mention affected resource as terraform. Example: affected_resources = terraform

As this works through automation, request you to follow exact syntax.

[steve-gray]

Are you being real? This a is a super simple concept - you literally can’t issue a Kubernetes token for OKE via terraform to use with other providers in the same plan.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: tf-oci-pub @.> Sent: Thursday, May 4, 2023 7:16:51 PM To: oracle/terraform-provider-oci @.> Cc: Steve Gray @.>; Author @.> Subject: Re: [oracle/terraform-provider-oci] Suport for Nested Helm/Kubernetes Providers on Terraform Cloud (Issue #1489)

Thank you for reporting the issue. We observed the affected resources are not provided in the description or it's incorrect. We request you to add it in issue description as mentioned in below format. Example: affected_resources = oci_core_instance , oci_core_instances

If it's not related to any particular resource then mention affected resource as terraform. Example: affected_resources = terraform

As this works through automation, request you to follow exact syntax.

— Reply to this email directly, view it on GitHubhttps://github.com/oracle/terraform-provider-oci/issues/1489#issuecomment-1534371679, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADPDTPZKPMWFM3W73MCOGRLXENXYHANCNFSM5I5B64KA. You are receiving this because you authored the thread.Message ID: @.***>

[tf-oci-pub]

Thank you for reporting the issue. We have raised an internal ticket to track this. Our service engineers will get back to you.

[XinruXiao-9]

Hi @steve-gray , we supported using data source to generate kubeconfig via terraform. you can use yaml decode to support helm/kuberneters provider.

data "oci_containerengine_cluster_kube_config" "oke_cluster_kube_config" {
  cluster_id = var.cluster_id
}
provider "kubernetes" {
  host                   = yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["clusters"][0]["cluster"]["server"]
  cluster_ca_certificate = base64decode(yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["clusters"][0]["cluster"]["certificate-authority-data"])

  exec {
    api_version = yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["apiVersion"]
    args = [yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["args"][0],
      yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["args"][1],
      yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["args"][2],
      yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["args"][3],
      yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["args"][4],
      yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["args"][5],
      yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["args"][6]]
    command = yamldecode(data.oci_containerengine_cluster_kube_config.oke_cluster_kube_config.content)["users"][0]["user"]["exec"]["command"]
  }
}

Or you can also save it as a resource using

data "oci_containerengine_cluster_kube_config" "cluster_kube_config" {
   cluster_id = var.cluster_id
}
resource "local_file" "kube_config_file" {
  content  = data.oci_containerengine_cluster_kube_config.cluster_kube_config.content
  filename = var.filename
}

[tf-oci-pub]

Fixed in version 4.123.0 by @XinruXiao-9

[steve-gray]

If this requires the OCI tools installed, it's not a solution to the issue we are having @XinruXiao-9.

The generated kubeconfig from those resources references the CLI Authenticator, whereas for EKS/AKS/GKE it's possible to get a specific token to use.