talos
talos copied to clipboard
Unable to upgrade k8s to 1.30.1
Bug Report
I'm running a two node Talos cluster on Proxmox - one control plane node and one worker node. When attempting to upgrade Kubernetes from 1.30.0 to 1.30.1, the upgrade always fails. I've attempted this upgrade when booted from Talos 1.7.0, 1.7.1, 1.7.2, and 1.7.4. Each time, the upgrade times out when attempting to upgrade kubelet on the control plane node.
Logs
$ talosctl upgrade-k8s
automatically detected the lowest Kubernetes version 1.30.0
discovered controlplane nodes ["192.168.10.11" "192.168.10.11"]
discovered worker nodes ["192.168.10.12"]
> "192.168.10.11": pre-pulling registry.k8s.io/kube-apiserver:v1.30.1
> "192.168.10.11": pre-pulling registry.k8s.io/kube-apiserver:v1.30.1
> "192.168.10.11": pre-pulling registry.k8s.io/kube-controller-manager:v1.30.1
> "192.168.10.11": pre-pulling registry.k8s.io/kube-controller-manager:v1.30.1
> "192.168.10.11": pre-pulling registry.k8s.io/kube-scheduler:v1.30.1
> "192.168.10.11": pre-pulling registry.k8s.io/kube-scheduler:v1.30.1
> "192.168.10.11": pre-pulling ghcr.io/siderolabs/kubelet:v1.30.1
> "192.168.10.11": pre-pulling ghcr.io/siderolabs/kubelet:v1.30.1
> "192.168.10.12": pre-pulling ghcr.io/siderolabs/kubelet:v1.30.1
updating "kube-apiserver" to version "1.30.1"
> "192.168.10.11": starting update
> "192.168.10.11": machine configuration patched
> "192.168.10.11": waiting for kube-apiserver pod update
< "192.168.10.11": successfully updated
> "192.168.10.11": starting update
> "192.168.10.11": machine configuration patched
> "192.168.10.11": waiting for kube-apiserver pod update
< "192.168.10.11": successfully updated
updating "kube-controller-manager" to version "1.30.1"
> "192.168.10.11": starting update
> "192.168.10.11": machine configuration patched
> "192.168.10.11": waiting for kube-controller-manager pod update
> "192.168.10.11": kube-controller-manager: pod is not ready, waiting
> "192.168.10.11": kube-controller-manager: pod is not ready, waiting
< "192.168.10.11": successfully updated
> "192.168.10.11": starting update
> "192.168.10.11": machine configuration patched
> "192.168.10.11": waiting for kube-controller-manager pod update
< "192.168.10.11": successfully updated
updating "kube-scheduler" to version "1.30.1"
> "192.168.10.11": starting update
> "192.168.10.11": machine configuration patched
> "192.168.10.11": waiting for kube-scheduler pod update
< "192.168.10.11": successfully updated
> "192.168.10.11": starting update
> "192.168.10.11": machine configuration patched
> "192.168.10.11": waiting for kube-scheduler pod update
< "192.168.10.11": successfully updated
updating kube-proxy to version "1.30.1"
> "192.168.10.11": starting update
> "192.168.10.11": starting update
updating kubelet to version "1.30.1"
> "192.168.10.11": starting update
> "192.168.10.11": machine configuration patched
> "192.168.10.11": waiting for node update
failed upgrading kubelet: error updating node "192.168.10.11": 3 error(s) occurred:
node version mismatch: got "v1.30.0", expected "v1.30.1"
etcdserver: request timed out
timeout
Environment
- Talos version:
$ talosctl version -n 192.168.10.11,192.168.10.12
Client:
Tag: v1.7.4
SHA: cb3a8308
Built:
Go version: go1.22.3
OS/Arch: darwin/arm64
Server:
NODE: 192.168.10.11
Tag: v1.7.4
SHA: cb3a8308
Built:
Go version: go1.22.3
OS/Arch: linux/amd64
Enabled: RBAC
NODE: 192.168.10.12
Tag: v1.7.4
SHA: cb3a8308
Built:
Go version: go1.22.3
OS/Arch: linux/amd64
Enabled: RBAC
- Kubernetes version:
$ kubectl version
Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.1
- Platform: metal-amd64