vcluster
vcluster copied to clipboard
loft-sh
out-kube-config-secret-namespace feature produces unknown namespace for the cache error
What happened?
I try to use out-kube-config-secret-namespace syncer feature but I'm getting this error:
Error writing kube config to secret: creating xxxxx-kubeconfig secret in the vclusters ns failed: apply generated kube config secret: unable to get: vclusters/xxxxx-kubeconfig because of unknown namespace for the cache
vCluster syncer is running in vcluster-xxxx namespace and I would like my kubeconfig secret to be available in vclusters namespace. I added this RBAC so syncer pod can read/write secrets in vclusters namespace without luck. Any idea what's wrong ?
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: vclusters-kubeconfig-rw
namespace: vclusters
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: vclusters-kubeconfig-rw-xxxxx
namespace: vclusters
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: vclusters-kubeconfig-rw
subjects:
- kind: ServiceAccount
name: vc-vcluster
namespace: vcluster-xxxx
What did you expect to happen?
- No longer syncer error and a secret write in the right namespace
- Minimal RBAC documented for this feature
How can we reproduce it (as minimally and precisely as possible)?
Just use out-kube-config-secret-namespace feature with a namespace different than vcluster namespace
Anything else we need to know?
No response
Host cluster Kubernetes version
$ kubectl version
1.28
Host cluster Kubernetes distribution
EKS
vlcuster version
$ vcluster --version
v0.16.2
Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)
# Write here
OS and Arch
OS:
Arch:
I've hit this problem as well, seems that in the current version of controller-runtime it is entering multi-namespace mode, and when creating the blocking cache client if the separate kubeconfig secret namespace is not in the list of default namespaces no namespace cache mapping is initialized
I'd suggest creating a new client to interact with the separate namespace to manage the kubeconfig secret, unless there's a way to do it via the existing blocking cache client? If the separate namespace is added to the default namespaces of the existing client it probably will initialize informers for a lot more than the kubeconfig secret
@rikycaldeira sounds like you understand the problem quite well, would you be interested in contributing a PR? And since it is Hacktoberfest, Loft is also giving away some swag for the PR contributors. :)
I can give it a try, although I'm not super familiar with the codebase, hope I don't ruin something :D I'll follow my own suggestion of instantiating a dedicated client and put it up for review
Awesome! Feel free to reach out to the team in Slack if you need any help. Don't forget to check out contributing.md, we are using DevSpace for dev, which should make iterating the implementation easier :)