kubernetes
Feature Request: Support $HOME/.kube/config.d/*
Greetings!
kubectl already allows defining multiple clusters and users in $HOME/.kube/config however editing this file by hand or even by tools is a bit cumbersome.
If kubectl supported loading multiple config files from $HOME/.kube/config.d/ it would make dealing with different cluster configurations much easier.
For example,
-
kubespray already generates a config file, but it is still not easy to use it if you already have a config file setup (kubernetes-sigs/kubespray/pull/1647).
-
aws eks cli already mutates the config file, but dealing with multiple clusters or updating this information requires way more mental overhead than it should require.
I would like to hear some feedback on this and whatever a pull request would be considered.
Many Thanks!
Can't you do the same by having multiple config files in arbitrary locations and listing them in the KUBECONFIG environment variable?
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
I'd be keen on this too.; If there are others who would like it then would be happy to put some time in for a PR
@weibeld Sure, but that wouldn't be the standard way and will be hard to convince tool vendors to follow through, it is also not exactly simple or easy approach.
Sure, a $HOME/.kube/config.d/ would probably be more convenient, I just think maybe the Kubernetes maintainers might not want to introduce it, because they created the KUBECONFIG variable for the purpose of having multiple separate kubeconfig files.
What do you mean by "not the standard way"?
I think KUBECONFIG has it's own valid use, as means of overriding the default or active switch/context.
However, if kubectl context worked with $HOME/.kube/config.d/ files, tools (terraform, eks cli, kubespray, whatnot!) would emit their config files there with an appropriate name instead of trying to mutate $HOME/.kube/config (eks cli, atm), or worst overwrite it, or possibly leave it up to the user which has a cumbersome ux (what kubespray does atm).
What I mean by standard way is, asking a tool maintainer to support kubectl context is much more convincing than asking them to support some ad-hoc workflow.
Hope that makes it clear :)
Is kubectl-switch a specific kubectl plugin or command?
In general, I wouldn't edit a kubeconfig file by hand, but use the kubectl config sub-commands. And for common tasks like changing the current context, or changing the namespace of the current context, I would use a tool like kubectx or something like these kubectl-ctx and kubectl-ns plugins.
Is kubectl-switch a specific kubectl plugin or command?
By kubectl-switch, I meant kubectl config set-context.
@weibeld I would tend to automate using kubectl config too, but for the sake of organisation I can still imagine users who would like to arrange their own config files in a more structured manner, a bit like in a sources.d/ for apt. Even with kubectl config being able to specify that the config is written to a file inside of the config.d directory might be nice, meaning that you can automate the addition of configuration, but still ensure it ends up in a well organised location.
I have a few use cases for this, one is something like:
https://github.com/dwmkerr/terraform-aws-openshift
Where I'd like to run make infra to create the infrastructure, then make openshift to create openshift setup, but ideally then run make kube-config or something to create the config file. But ideally I'd love to keep this config separate from my main 'working' config.
@dwmkerr It could indeed be more user-friendly to replace the KUBECONFIG variable with a .kube/config.d directory. So that the effective configuration would always be the merge of the default .kube/config file and all files in .kube/config.d (if present). The KUBECONFIG variable wouldn't be needed anymore. Or does someone see a use case where this approach is inferior to having the KUBEONFIG variable?
In the end, both approaches allow to do the same, but the directory approach frees you from having to deal with environment variables (which can be accidentally set and unset easily).
Currently, in your case, you would need to have your make kube-config create a .kube/aws-openshift config file for your cluster, and then set KUBECONFIG=~/.kube/config:~/.kube/aws-openshift. To make it persistent, you would have to add this to your ~/.bashrc, and then you have to hope that KUBECONFIG never gets changed by accident. So, yeah, just dropping the file into ~/.kube/config.d would definitely be easier. If you want to do a PR, it would certainly be a useful feature.
FYI I've created the design document and raised the feature request in the appropriate location (I think!) and mentioned this issue there, so I believe this issue can be closed now as it is linked to be the new feature request.
https://github.com/kubernetes/kubernetes/issues/80120
I closed this issue under the impression that you have opened a pull request, now after some looking into this again, I am not sure that is the case. Is this issue not the right way to go about discussion whatever something is up for consideration or not?
I don't understand why an issue related to kubectl needs to be opened against kubernetes or community. Can you please help me understand?
Hi @omeid honest I found the guide for contributing quite confusing, I was following the instructions here:
https://github.com/kubernetes/community/blob/master/sig-cli/CONTRIBUTING.md
I followed the instructions which involved creating the design doc, opening an issue on staging etc, it's quite a complex process so far, and I'm not sure the issues are even being seen...
@dwmkerr if you're not getting responses from sig-cli, they list ways you can escalate for attention: https://github.com/kubernetes/community/blob/master/sig-cli/CONTRIBUTING.md#general-escalation-instructions
Hi @cblecker thanks for the tips! Just pinged on the group, I've tried slack with no joy, but let's see what the group says. Really appreciate the help!
There's a lot of history here: https://github.com/kubernetes/kubernetes/issues/9298 https://github.com/kubernetes/kubernetes/issues/10693 https://github.com/kubernetes/kubernetes/issues/20605 https://github.com/kubernetes/kubernetes/issues/30395 https://github.com/kubernetes/kubernetes/pull/38861 https://github.com/kubernetes/kubernetes/issues/46381
And it would probably need to be respected by API client libraries, not just kubectl.
Hi @seans3 - thanks I saw your message for the SIG meeting, I'll join in the discussion! Yes I hadn't realised just quite how much history there is, wow!
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
Is this answer sufficient: https://stackoverflow.com/questions/46184125/how-to-merge-kubectl-config-file-with-kube-config/46184649#46184649 ?
