kubekey 2.1 create cluster --with-kubernetes 1.21.13 ERROR

[Heyz6]

What is version of KubeKey has the issue?

2.1.0

What is your os environment?

centos 7.9

KubeKey config file

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: sample
spec:
  hosts:
  - {name: node1, address: "192.168.66.66", internalAddress: "192.168.66.66", port: 22, user: "root", password: "password"}
  roleGroups:
    etcd:
    - node1
    control-plane:
    - node1
    worker:
    - node1
  controlPlaneEndpoint:
    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.12.13
    clusterName: cluster.local
    autoRenewCerts: true
  etcd:
    type: kubekey
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
    multusCNI:
      enabled: false
  registry:
    privateRegistry: ""
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
  addons: []

A clear and concise description of what happend.

export KKZONE=cn;./kk create cluster --skip-pull-images --with-kubernetes v1.21.13 -y -f ./config-sample.yaml

---

| | / / | | | | / / | |/ / _ | |_ | |/ / ___ _ _ | | | | | ' \ / _ \ \ / _ \ | | | | |\ \ || | |) | / |\ \ / || | _| _/_,|./ __| _/___|_, | / | |/

14:35:13 CST [GreetingsModule] Greetings 14:35:13 CST message: [node1] Greetings, KubeKey! 14:35:13 CST success: [node1] 14:35:13 CST [NodePreCheckModule] A pre-check on nodes 14:35:13 CST success: [node1] 14:35:13 CST [NodeBinariesModule] Download installation binaries 14:35:13 CST message: [localhost] downloading amd64 kubeadm v1.21.13 ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42.8M 100 42.8M 0 0 1032k 0 0:00:42 0:00:42 --:--:-- 1180k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42.8M 100 42.8M 0 0 1016k 0 0:00:43 0:00:43 --:--:-- 1053k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42.8M 100 42.8M 0 0 1035k 0 0:00:42 0:00:42 --:--:-- 1188k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42.8M 100 42.8M 0 0 1034k 0 0:00:42 0:00:42 --:--:-- 1201k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42.8M 100 42.8M 0 0 1018k 0 0:00:43 0:00:43 --:--:-- 1052k 14:38:47 CST message: [LocalHost] Failed to download kubeadm binary: curl -L -o /opt/kubekey/kube/v1.21.13/amd64/kubeadm https://kubernetes-release.pek3b.qingstor.com/release/v1.21.13/bin/linux/amd64/kubeadm error: No SHA256 found for kubeadm. v1.21.13 is not supported. 14:38:47 CST failed: [LocalHost] error: Pipeline[CreateClusterPipeline] execute failed: Module[NodeBinariesModule] exec failed: failed: [LocalHost] [DownloadBinaries] exec failed after 1 retires: Failed to download kubeadm binary: curl -L -o /opt/kubekey/kube/v1.21.13/amd64/kubeadm https://kubernetes-release.pek3b.qingstor.com/release/v1.21.13/bin/linux/amd64/kubeadm error: No SHA256 found for kubeadm. v1.21.13 is not supported.

Relevant log output

cat kubekey/logs/kubekey.log
14:35:13 CST [GreetingsModule] Greetings
14:35:13 CST message: [node1]
Greetings, KubeKey!
14:35:13 CST success: [node1]
14:35:13 CST [NodePreCheckModule] A pre-check on nodes
14:35:13 CST success: [node1]
14:35:13 CST [NodeBinariesModule] Download installation binaries
14:35:13 CST message: [localhost]
downloading amd64 kubeadm v1.21.13 ...
14:38:47 CST message: [LocalHost]
Failed to download kubeadm binary: curl -L -o /opt/kubekey/kube/v1.21.13/amd64/kubeadm https://kubernetes-release.pek3b.qingstor.com/release/v1.21.13/bin/linux/amd64/kubeadm error: No SHA256 found for kubeadm. v1.21.13 is not supported.

Additional information

no

[24sama]

Hi @Heyz6 We are going to release a new version. And we just modified the readme. So if you want to create a v1.21.13 k8s cluster. You can build the kk by yourself.

git clone https://github.com/kubesphere/kubekey.git
cd kubekey
make kk

Or, wait for kk new version.

[Heyz6]

go fmt ./... go: downloading k8s.io/apimachinery v0.23.6 go: downloading k8s.io/client-go v0.23.6 go: downloading github.com/pkg/errors v0.9.1 go: downloading sigs.k8s.io/controller-runtime v0.11.2 go: downloading github.com/spf13/cobra v1.4.0 go: downloading k8s.io/kubectl v0.23.6 go: downloading github.com/lithammer/dedent v1.1.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/modood/table v0.0.0-20200225102042-88de94bb9876 go: downloading github.com/go-logr/logr v1.2.3 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading k8s.io/api v0.23.6 go: downloading github.com/deckarep/golang-set v1.8.0 go: downloading helm.sh/helm/v3 v3.7.2 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/dominodatalab/os-release v0.0.0-20190522011736-bcdb4a3e3c2f go: downloading github.com/spf13/viper v1.11.0 go: downloading k8s.io/utils v0.0.0-20211116205334-6203023598ed go: downloading github.com/pkg/sftp v1.13.4 go: downloading github.com/lestrrat-go/file-rotatelogs v2.4.0+incompatible go: downloading golang.org/x/crypto v0.0.0-20220411220226-7b82a4e95df4 go: downloading github.com/rifflock/lfshook v0.0.0-20180920164130-b9218ef580f5 go: downloading k8s.io/cli-runtime v0.23.6 go: downloading github.com/sirupsen/logrus v1.8.1 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/containers/image/v5 v5.21.1 go: downloading github.com/estesp/manifest-tool/v2 v2.0.3 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20211202193544-a5463b7f9c84

sorry,When I got to this point I waited a long time and retried three times, but I was not successful. @24sama

[24sama]

@Heyz6 kk v2.2.0 is already released. You can have a try.

[Heyz6]

thank you.Installation succeeded @ @24sama

[bashfulrobot]

Hi there, I am running into this now as well.

Version

❯ kk version
version.BuildInfo{Version:"2.2.1", GitCommit:"c056977c", GitTreeState:"", GoVersion:"go1.17.11"}

Deploy Command

kk create cluster -f dk-kk-config.yaml

Error

error: Pipeline[CreateClusterPipeline] execute failed: Module[NodeBinariesModule] exec failed: 
failed: [LocalHost] [DownloadBinaries] exec failed after 1 retires: Failed to download kubeadm binary: curl -L -o /home/dustin/.kube/kk/kubekey/kube/1.24.1/amd64/kubeadm https://storage.googleapis.com/kubernetes-release/release/1.24.1/bin/linux/amd64/kubeadm error: No SHA256 found for kubeadm. 1.24.1 is not supported.

dk-kk-config.yaml

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: kk-dt
spec:
  hosts:
  - {name: kkc1, address: 192.168.122.110, internalAddress: 192.168.122.110, user: dustin, password: "xxxxx", privateKeyPath: "/home/dustin/.ssh/id_rsa_temp"}
  - {name: kkn1, address: 192.168.122.26, internalAddress: 192.168.122.26, user: dustin, password: "xxxxx", privateKeyPath: "/home/dustin/.ssh/id_rsa_temp"}
  - {name: kkn2, address: 192.168.122.132, internalAddress: 192.168.122.132, user: dustin, password: "xxxxx", privateKeyPath: "/home/dustin/.ssh/id_rsa_temp"}
  roleGroups:
    etcd:
    - kkc1
    control-plane:
    - kkc1
    worker:
    - kkn1
    - kkn2
  controlPlaneEndpoint:
    ## Internal loadbalancer for apiservers
    # internalLoadbalancer: haproxy

    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: 1.24.1
    clusterName: cluster.local
    autoRenewCerts: true
    containerManager: containerd
  etcd:
    type: kubekey
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
    ## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
    multusCNI:
      enabled: false
  registry:
    privateRegistry: ""
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
  addons: []

Am I missing something here?

Thanks, for your time.

[24sama]

hi @bashfulrobot I think it should be the v1.24.1 rather than 1.24.1

[bashfulrobot]

You are bang on!

What happened is when I ran the command to create the template file, I ran:

❯ kk create config --with-kubernetes 1.24.1 -f dk-kk-config.yaml

What I would kindly suggest is that there should be some sort of validation when someone runs the command if the "v" is needed.

Or I need to read the docs closer. 🤣

Cheers, — Dustin

On Fri, Jun 24, 2022 at 8:45 PM Leo Li @.***> wrote:

hi @bashfulrobot https://github.com/bashfulrobot I think it should be the v1.24.1 rather than 1.24.1

— Reply to this email directly, view it on GitHub https://github.com/kubesphere/kubekey/issues/1331#issuecomment-1166184012, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACSIRILX7SMDLRDGLIIQMR3VQZ6GLANCNFSM5YMTPXUA . You are receiving this because you were mentioned.Message ID: @.***>

[24sama]

You are bang on!

What happened is when I ran the command to create the template file, I ran:

❯ kk create config --with-kubernetes 1.24.1 -f dk-kk-config.yaml

What I would kindly suggest is that there should be some sort of validation when someone runs the command if the "v" is needed.

Or I need to read the docs closer. 🤣

Thanks for your suggestion! @bashfulrobot A semantic version with a "v" is just a common way (vA.B.C), and we also just followed Kubernetes. So, I think we can give some tips or auto-complete the version when the "v" is missing.

[xiaods]

--with-kubernetes only support prefix v + k8s version, such v1.21.14. please follow docs to install. I think this is not bug. don't need update current design.

[WangZhouX]

duubee@duubee7:~$ sudo ./kk create cluster --with-kubernetes v1.22.10 --with-kubesphere v3.3.0

---

| | / / | | | | / / | |/ / _ | |_ | |/ / ___ _ _ | | | | | ' \ / _ \ \ / _ \ | | | | |\ \ || | |) | / |\ \ / || | _| _/_,|./ __| _/___|_, | / | |/

23:34:35 CST [GreetingsModule] Greetings 23:34:35 CST failed: [duubee7] error: Pipeline[CreateClusterPipeline] execute failed: Module[GreetingsModule] exec failed: failed: [duubee7] failed to connect to 10.0.80.7: Failed to validate ssh connection parameters: Failed to read keyfile "/root/.ssh/id_rsa": open /root/.ssh/id_rsa: no such file or directory duubee@duubee7:~$ sudo ./kk create cluster --with-kubernetes v1.19.10 --with-kubesphere v3.3.0

为什么会这样 。。。。

[myltw]

@WangZhouX 你解决了吗

[xiaods]

duubee@duubee7:~$ sudo ./kk create cluster --with-kubernetes v1.22.10 --with-kubesphere v3.3.0

| | / / | | | | / / | |/ / _ | |_ | |/ / ___ _ _ | | | | | ' \ / _ \ \ / _ \ | | | | |\ \ || | |) | / |\ \ / |_| | | /__,|./ _| /|_, | / | |_/

23:34:35 CST [GreetingsModule] Greetings 23:34:35 CST failed: [duubee7] error: Pipeline[CreateClusterPipeline] execute failed: Module[GreetingsModule] exec failed: failed: [duubee7] failed to connect to 10.0.80.7: Failed to validate ssh connection parameters: Failed to read keyfile "/root/.ssh/id_rsa": open /root/.ssh/id_rsa: no such file or directory duubee@duubee7:~$ sudo ./kk create cluster --with-kubernetes v1.19.10 --with-kubesphere v3.3.0

为什么会这样 。。。。

please use root to install