setup/controller_context.go:225 couldn't find virtual cluster kube-config

[Rockyjr-git]

What happened?

vCluster Pods are not running, Deployed in the Anthos Bare Metal Cluster using Helm Chart.

kubectl get pods -n vcluster-dev
NAME             READY   STATUS    RESTARTS      AGE
vcluster-dev-0   0/1     Running   1 (20m ago)   53m

Below are the logs of vcluster Pod.

setup/initialize.go:88  failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-m2p76" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")        {"component": "vcluster-pro"}
2024-04-18 07:54:18     INFO    setup/controller_context.go:225 couldn't find virtual cluster kube-config, will retry in 1 seconds      {"component": "vcluster-pro"}
2024-04-18 07:54:19     INFO    setup/controller_context.go:225 couldn't find virtual cluster kube-config, will retry in 1 seconds      {"component": "vcluster-pro"}

What did you expect to happen?

vcluster and core dns pods are expected to run.

How can we reproduce it (as minimally and precisely as possible)?

Used to deploy k8s VMs in the vCluster.

Anything else we need to know?

No response

Host cluster Kubernetes version

Client Version: v1.27.3 Kustomize Version: v5.0.1 Server Version: v1.28.5-gke.1200

Host cluster Kubernetes distribution

# Anthos on Bare Metal Cluster

vlcuster version

$ vcluster --version
# v0.19.4

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

# Tried with k3s and k8s.

OS and Arch

OS: ubuntu 20. 04 .6 LTS ( Focal Fossa )
Arch: 

[FabianKramm]

@Rockyjr-git thanks for creating this issue! Is this a IPv6 only cluster?

[Rockyjr-git]

Yes, it is.

[janlauber]

We don't have the problem with k0sproject/k0s:v1.28.8-k0s.0 but when upgrading to k0sproject/k0s:v1.29.4-k0s.0 the following logs show up in the syncer (0.18.1 or 0.19.5) container:

2024-05-18 17:26:21	INFO	setup/initialize.go:86	k0s config secret detected, syncer will ensure that it contains service CIDR	{"component": "vcluster"}
2024-05-18 17:26:21	INFO	servicecidr/servicecidr.go:117	failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-jmp96" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")	{"component": "vcluster"}
2024-05-18 17:26:21	INFO	k0s/k0s.go:68	Starting k0s	{"component": "vcluster", "args": "/k0s-binary/k0s controller --config=/etc/k0s/config.yaml --data-dir=/data/k0s --status-socket=/run/k0s/status.sock --disable-components=konnectivity-server,kube-scheduler,csr-approver,kube-proxy,coredns,network-provider,helm,metrics-server,kubelet-config"}
2024-05-18 17:26:21	INFO	setup/controller_context.go:221	couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds	{"component": "vcluster"}
2024-05-18 17:26:21	INFO	setup/controller_context.go:221	couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds	{"component": "vcluster"}
2024-05-18 17:26:21	INFO	loghelper/klog.go:24	Error: unknown component kubelet-config	{"component": "vcluster", "component": "k0s"}
2024-05-18 17:26:21	INFO	setup/initialize.go:107	Error running k0s: exit status 1	{"component": "vcluster"}
2024-05-18 17:26:49	INFO	setup/initialize.go:86	k0s config secret detected, syncer will ensure that it contains service CIDR	{"component": "vcluster"}
2024-05-18 17:26:49	INFO	servicecidr/servicecidr.go:117	failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-zv7pg" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")	{"component": "vcluster"}
2024-05-18 17:26:49	INFO	k0s/k0s.go:68	Starting k0s	{"component": "vcluster", "args": "/k0s-binary/k0s controller --config=/etc/k0s/config.yaml --data-dir=/data/k0s --status-socket=/run/k0s/status.sock --disable-components=konnectivity-server,kube-scheduler,csr-approver,kube-proxy,coredns,network-provider,helm,metrics-server,kubelet-config"}
2024-05-18 17:26:49	INFO	setup/controller_context.go:221	couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds	{"component": "vcluster"}
2024-05-18 17:26:49	INFO	setup/controller_context.go:221	couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds	{"component": "vcluster"}
2024-05-18 17:26:49	INFO	loghelper/klog.go:24	Error: unknown component kubelet-config	{"component": "vcluster", "component": "k0s"}
2024-05-18 17:26:49	INFO	setup/initialize.go:107	Error running k0s: exit status 1	{"component": "vcluster"}
2024-05-18 17:27:33	INFO	setup/initialize.go:86	k0s config secret detected, syncer will ensure that it contains service CIDR	{"component": "vcluster"}
2024-05-18 17:27:33	INFO	servicecidr/servicecidr.go:117	failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-ftlgh" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")	{"component": "vcluster"}
2024-05-18 17:27:33	INFO	k0s/k0s.go:68	Starting k0s	{"component": "vcluster", "args": "/k0s-binary/k0s controller --config=/etc/k0s/config.yaml --data-dir=/data/k0s --status-socket=/run/k0s/status.sock --disable-components=konnectivity-server,kube-scheduler,csr-approver,kube-proxy,coredns,network-provider,helm,metrics-server,kubelet-config"}
2024-05-18 17:27:33	INFO	setup/controller_context.go:221	couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds	{"component": "vcluster"}
2024-05-18 17:27:33	INFO	setup/controller_context.go:221	couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds	{"component": "vcluster"}
2024-05-18 17:27:33	INFO	loghelper/klog.go:24	Error: unknown component kubelet-config	{"component": "vcluster", "component": "k0s"}
2024-05-18 17:27:33	INFO	setup/initialize.go:107	Error running k0s: exit status 1	{"component": "vcluster"}

[janekbaraniewski]

Hey @janlauber @Rockyjr-git ! Are you still experiencing this issue? If so, could you provide more details of your configuration? vcluster.yaml file used for installation would be a great start, also your network configuration and anthos deployment details.