k0sctl icon indicating copy to clipboard operation
k0sctl copied to clipboard

Published 20 hours ago • verified publisher icon k0sproject

Setup cluster with enabled dual stack not working

Open jasase opened this issue 2 years ago • 3 comments

Setup a fresh cluster with k0sctl with activated dual stack, like described in the documentation https://docs.k0sproject.io/v0.11.1/dual-stack/ results in a not working cluster.

Config to setup cluster:

apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
    - ssh:
        address: n1-k0s.example.conm
        user: sascha
        keyPath: /home/user1/.ssh/key
      role: controller+worker
      noTaints: true 
  k0s:
    version: 1.26.1+k0s.0
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: Cluster
      metadata:
        name: k0s-cluster
      spec:
        network:
          podCIDR: "10.244.0.0/16"
          serviceCIDR: "10.96.0.0/12"
          calico:
            mode: "bird"
          dualStack:
            enabled: true
            IPv6podCIDR: "f50::/108"
            IPv6serviceCIDR: "fd51::/108"

Cluster can not start metric and dns Pods:

kubectl get all -A
NAMESPACE     NAME                                  READY   STATUS             RESTARTS     AGE
kube-system   pod/coredns-9864b985-f2hgh            0/1     Running            0            74s
kube-system   pod/konnectivity-agent-rszxl          1/1     Running            0            63s
kube-system   pod/kube-proxy-wmbj4                  1/1     Running            0            65s
kube-system   pod/kube-router-8pmjz                 1/1     Running            0            66s
kube-system   pod/metrics-server-7446cc488c-kgv54   0/1     CrashLoopBackOff   2 (5s ago)   64s

NAMESPACE     NAME                     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
default       service/kubernetes       ClusterIP   10.96.0.1      <none>        443/TCP                  92s
kube-system   service/kube-dns         ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP,9153/TCP   80s
kube-system   service/metrics-server   ClusterIP   10.101.68.92   <none>        443/TCP                  65s

NAMESPACE     NAME                                DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/konnectivity-agent   1         1         1       1            1           kubernetes.io/os=linux   80s
kube-system   daemonset.apps/kube-proxy           1         1         1       1            1           kubernetes.io/os=linux   85s
kube-system   daemonset.apps/kube-router          1         1         1       1            1           <none>                   80s

NAMESPACE     NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/coredns          0/1     1            0           80s
kube-system   deployment.apps/metrics-server   0/1     1            0           65s

NAMESPACE     NAME                                        DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/coredns-9864b985            1         1         0       75s
kube-system   replicaset.apps/metrics-server-7446cc488c   1         1         0       64s

Seems that the kubernetes API is not reachable inside of the cluster:

Log of CoreDNS:

2023-02-26T13:24:41.742108665Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:42.13019413Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:42.240988081Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:42.630191343Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:42.744782848Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:43.129889565Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:43.629279106Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:43.746767681Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:44.129292245Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:44.240646999Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:44.629419398Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:45.129500456Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:45.632404395Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:46.129699607Z stdout F [INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
2023-02-26T13:24:46.240867972Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:46.629981898Z stdout F [WARNING] plugin/kubernetes: starting server with unsynced Kubernetes API
2023-02-26T13:24:46.630310794Z stdout F .:53
2023-02-26T13:24:46.630366793Z stdout F [INFO] plugin/reload: Running configuration SHA512 = a0c06adb3aa41fc5baa9a525a356d4f8e0206ad810f6a0d1f005e63150f4cd85bb682bac617e725b3996ba493aee5bd94a076b3ff9c64726e7e00a033b2ab1ba
2023-02-26T13:24:46.630374593Z stdout F CoreDNS-1.10.0
2023-02-26T13:24:46.630378993Z stdout F linux/amd64, go1.19.1, 596a9f9
2023-02-26T13:24:48.240560411Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:50.241073398Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:51.629968058Z stdout F [WARNING] plugin/kubernetes: Kubernetes API connection failure: Get "https://10.96.0.1:443/version": dial tcp 10.96.0.1:443: connect: network is unreachable
2023-02-26T13:24:52.241264244Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:52.639132358Z stdout F [ERROR] plugin/errors: 2 3405729959988026839.47239424169260323. HINFO: read udp [f50::4]:33604->[fd01::36]:53: i/o timeout
2023-02-26T13:24:53.63439146Z stdout F [ERROR] plugin/errors: 2 3405729959988026839.47239424169260323. HINFO: dial udp 10.111.111.36:53: connect: network is unreachable
2023-02-26T13:24:54.240621752Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:54.639481385Z stdout F [ERROR] plugin/errors: 2 3405729959988026839.47239424169260323. HINFO: read udp [f50::4]:58702->[fd01::36]:53: i/o timeout
2023-02-26T13:24:55.640767991Z stdout F [ERROR] plugin/errors: 2 3405729959988026839.47239424169260323. HINFO: dial udp 10.111.111.36:53: connect: network is unreachable
2023-02-26T13:24:56.241290595Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:24:58.240909198Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:00.24110744Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:01.73425832Z stdout F [WARNING] plugin/kubernetes: Kubernetes API connection failure: Get "https://10.96.0.1:443/version": dial tcp 10.96.0.1:443: connect: network is unreachable
2023-02-26T13:25:02.241172826Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:04.240996456Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:06.241020323Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:08.241260224Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:10.241217863Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:12.134349217Z stdout F [WARNING] plugin/kubernetes: Kubernetes API connection failure: Get "https://10.96.0.1:443/version": dial tcp 10.96.0.1:443: connect: network is unreachable
2023-02-26T13:25:12.241164534Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:14.244656496Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:16.240970566Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:18.240910423Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:20.241110701Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:21.734179566Z stdout F [WARNING] plugin/kubernetes: Kubernetes API connection failure: Get "https://10.96.0.1:443/version": dial tcp 10.96.0.1:443: connect: network is unreachable
2023-02-26T13:25:22.241105403Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:24.248750544Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:26.245930117Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:28.24172302Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:30.241637394Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:31.734466299Z stdout F [WARNING] plugin/kubernetes: Kubernetes API connection failure: Get "https://10.96.0.1:443/version": dial tcp 10.96.0.1:443: connect: network is unreachable
2023-02-26T13:25:32.240551192Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:34.241494681Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:36.241102593Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:38.241569403Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:40.244288398Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:41.734375435Z stdout F [WARNING] plugin/kubernetes: Kubernetes API connection failure: Get "https://10.96.0.1:443/version": dial tcp 10.96.0.1:443: connect: network is unreachable
2023-02-26T13:25:42.250746859Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:44.244567752Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:46.24787815Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:48.240920451Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:50.241560374Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:51.733740863Z stdout F [WARNING] plugin/kubernetes: Kubernetes API connection failure: Get "https://10.96.0.1:443/version": dial tcp 10.96.0.1:443: connect: network is unreachable
2023-02-26T13:25:52.245759359Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:52.41030782Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:54.245365584Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"
2023-02-26T13:25:56.241351439Z stdout F [INFO] plugin/ready: Still waiting on: "kubernetes"

jasase avatar Feb 26 '23 13:02 jasase