kubeclient icon indicating copy to clipboard operation
kubeclient copied to clipboard

Make in-cluster configuration easy

Open cben opened this issue 6 years ago • 3 comments

Splitting from discussion in #394:

Tangentially, I would suggest that we configure this to "just work" inside a Kubernetes cluster. That is detect and implement the code in the README section "Inside a Kubernetes Cluster" when no other authorization is provided.


That sounds like a good idea. Looking at how client-go implements rest.InClusterConfig, looks like we can just lift that wholesale. In fact, that's pretty much what I've seen most apps doing (unsurprisingly).

I think adding the in-cluster ability as a new method on Kubeclient::Config should be sufficient, and we'll leave the responsibility of calling the appropriate method to the caller (since sometimes clients are run in-cluster, but with a mounted kubeconfig to speak with other clusters). This has the nice benefit of mirroring the conventions of the client-go package as well


For reference see also our gem that implements this (in ruby): https://github.com/keylimetoolbox/resque-kubernetes/blob/v2.0.0/lib/resque/kubernetes/context/well_known.rb

And yes, we should do this as a last resort. If there's a provided configuration use that; if not, check for GCP; final step is to check for in-cluster.

cben avatar Feb 24 '19 15:02 cben

BTW, there is a small discrepancy between what k8s documents and what client-go does, related to rare issue https://github.com/kubernetes/kubernetes/issues/40973 where the env vars are not always set. See previous discussion on #316.

cben avatar Feb 24 '19 17:02 cben

I think it'll make sense to extract config & auth parts from kubeclient + https://github.com/kubernetes-client/ruby/ into a new gem and make both projects use it. They already have InClusterConfig class, Kubernetes.load_incluster_config helper, and default logic in https://github.com/kubernetes-client/ruby/blob/master/kubernetes/src/kubernetes/loader.rb that tries:

  1. $KUBECONFIG, if file exists
  2. ~/.kube/config, if file exists
  3. InClusterConfig.in_cluster?

cben avatar May 01 '19 08:05 cben

Kubernetes plans to change service account tokens to short-lived: https://github.com/kubernetes/kubernetes/issues/70679 This even simple in-cluster config will require on-the fly auth updates (#393)

cben avatar Aug 07 '19 05:08 cben