oci-kubernetes-monitoring icon indicating copy to clipboard operation
oci-kubernetes-monitoring copied to clipboard

How to use this code with private endpoints

Open rhegde2 opened this issue 1 year ago • 4 comments

Hi, We tried to deploy it against an OKE running on private endpoint. We can't create public endpoint because of our company policies.

So, Resource manager stack is failing with the below error:

Error: Kubernetes cluster unreachable: Get "https://<PRIVATE_IP>:6443/version": dial tcp <PRIVATE_IP>:6443: i/o timeout

We tried with creating a reachable IP and no luck.

Kindly suggest.

rhegde2 avatar Jul 12 '23 07:07 rhegde2

The private endpoint support for installation through Resource Manager is not yet available, it would come in upcoming releases. You may try using the helm chart based installation as an alternative.

santhoshkvuda avatar Jul 13 '23 09:07 santhoshkvuda

hi @santhoshkvuda Thanks a lot for your inputs.

Any tentative date for the upcoming release?

rhegde2 avatar Jul 13 '23 11:07 rhegde2

One more thing to add is that even you create an oci_resourcemanager_private_endpoint and then use , and use it on your Help configuration for example like:

resource "oci_resourcemanager_private_endpoint" "rms_pe" {
  compartment_id = var.compartment_ocid
  display_name   = var. display_name
  description       =  var. description
  vcn_id              = var.vcn_id
  subnet_id        = var.subnet_id
}

And then get the Reachable IP from the datasource

data "oci_resourcemanager_private_endpoint_reachable_ip" "rms_pe_reachable_ip_address" {
  private_endpoint_id = oci_resourcemanager_private_endpoint.rms_pe.id
  private_ip          = "<OKE_API_endpoint>"
}
provider "helm" {
  kubernetes {
    host                   = "https://${data.oci_resourcemanager_private_endpoint_reachable_ip.rms_pe_reachable_ip_address.ip_address}:6443"
    cluster_ca_certificate = local.cluster_ca_certificate
    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      args        = ["ce", "cluster", "generate-token", "--cluster-id", local.cluster_id, "--region", local.cluster_region]
      command     = "oci"
    }
  }
}

You are able to reach out to the Cluster the OKE is still unreachable with the error:

Error: Kubernetes cluster unreachable: Get "https://<REACHABLE_IP>:6443/version": x509: certificate is valid for ...... 168.254.5.1, 127.0.0.1, not <REACHABLE_IP>

consiahras avatar Aug 01 '23 13:08 consiahras

I believe this article describes the fix https://docs.oracle.com/en/learn/logging-for-private-oke-cluster/index.html It didn't worked for me, but I believe it is my Private Endpoint setup issues. In general seem to be the fix for your problem.

Tyson1986 avatar Mar 12 '24 06:03 Tyson1986