Using sealos 5.0.0-beta5 cannot upgrade k8s from 1.27.9 to 1.28.9

[caijian76]

Sealos Version

5.0.0-beta5

How to reproduce the bug?

# sealos run registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.28.9
2024-05-10T01:20:01 info start to install app in this cluster
2024-05-10T01:20:01 info Executing SyncStatusAndCheck Pipeline in InstallProcessor
2024-05-10T01:20:01 info Executing ConfirmOverrideApps Pipeline in InstallProcessor
2024-05-10T01:20:01 info Executing PreProcess Pipeline in InstallProcessor
2024-05-10T01:20:01 info Executing pipeline MountRootfs in InstallProcessor.
2024-05-10T01:20:02 info Executing pipeline MirrorRegistry in InstallProcessor.
2024-05-10T01:20:02 info trying default http mode to sync images to hosts [192.168.138.129:22]
2024-05-10T01:20:05 info Executing UpgradeIfNeed Pipeline in InstallProcessor
2024-05-10T01:20:05 info Change ClusterConfiguration up to newVersion if need.
2024-05-10T01:20:05 info kubernetes client config rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"", KeyFile:"", CAFile:"", CertData:[]uint8{0x2d, 0x2d, 0x2d, 0x20, 0x54, 0x52, 0x55, 0x4e, 0x43, 0x41, 0x54, 0x45, 0x44, 0x20, 0x2d, 0x2d, 0x2d}, KeyData:[]uint8{0x2d, 0x2d, 0x2d, 0x20, 0x52, 0x45, 0x44, 0x41, 0x43, 0x54, 0x45, 0x44, 0x20, 0x2d, 0x2d, 0x2d}, CAData:[]uint8{0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0x42, 0x45, 0x47, 0x49, 0x4e, 0x20, 0x43, 0x45, 0x52, 0x54, 0x49, 0x46, 0x49, 0x43, 0x41, 0x54, 0x45, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0xa, 0x4d, 0x49, 0x49, 0x43, 0x36, 0x54, 0x43, 0x43, 0x41, 0x64, 0x47, 0x67, 0x41, 0x77, 0x49, 0x42, 0x41, 0x67, 0x49, 0x42, 0x41, 0x44, 0x41, 0x4e, 0x42, 0x67, 0x6b, 0x71, 0x68, 0x6b, 0x69, 0x47, 0x39, 0x77, 0x30, 0x42, 0x41, 0x51, 0x73, 0x46, 0x41, 0x44, 0x41, 0x56, 0x4d, 0x52, 0x4d, 0x77, 0x45, 0x51, 0x59, 0x44, 0x56, 0x51, 0x51, 0x44, 0x45, 0x77, 0x70, 0x72, 0x64, 0x57, 0x4a, 0x6c, 0xa, 0x63, 0x6d, 0x35, 0x6c, 0x64, 0x47, 0x56, 0x7a, 0x4d, 0x43, 0x41, 0x58, 0x44, 0x54, 0x49, 0x30, 0x4d, 0x44, 0x55, 0x77, 0x4f, 0x54, 0x45, 0x33, 0x4d, 0x54, 0x41, 0x7a, 0x4f, 0x46, 0x6f, 0x59, 0x44, 0x7a, 0x49, 0x78, 0x4d, 0x6a, 0x51, 0x77, 0x4e, 0x44, 0x45, 0x31, 0x4d, 0x54, 0x63, 0x78, 0x4d, 0x44, 0x4d, 0x34, 0x57, 0x6a, 0x41, 0x56, 0x4d, 0x52, 0x4d, 0x77, 0x45, 0x51, 0x59, 0x44, 0xa, 0x56, 0x51, 0x51, 0x44, 0x45, 0x77, 0x70, 0x72, 0x64, 0x57, 0x4a, 0x6c, 0x63, 0x6d, 0x35, 0x6c, 0x64, 0x47, 0x56, 0x7a, 0x4d, 0x49, 0x49, 0x42, 0x49, 0x6a, 0x41, 0x4e, 0x42, 0x67, 0x6b, 0x71, 0x68, 0x6b, 0x69, 0x47, 0x39, 0x77, 0x30, 0x42, 0x41, 0x51, 0x45, 0x46, 0x41, 0x41, 0x4f, 0x43, 0x41, 0x51, 0x38, 0x41, 0x4d, 0x49, 0x49, 0x42, 0x43, 0x67, 0x4b, 0x43, 0x41, 0x51, 0x45, 0x41, 0xa, 0x78, 0x77, 0x32, 0x33, 0x2f, 0x5a, 0x46, 0x74, 0x4d, 0x47, 0x42, 0x4c, 0x46, 0x72, 0x6c, 0x43, 0x31, 0x68, 0x36, 0x4f, 0x77, 0x58, 0x78, 0x4f, 0x4f, 0x32, 0x6f, 0x52, 0x35, 0x68, 0x7a, 0x77, 0x61, 0x4f, 0x57, 0x78, 0x53, 0x31, 0x6c, 0x6e, 0x4d, 0x66, 0x4d, 0x46, 0x31, 0x65, 0x72, 0x39, 0x5a, 0x69, 0x77, 0x33, 0x4c, 0x39, 0x58, 0x51, 0x4e, 0x52, 0x57, 0x59, 0x7a, 0x31, 0x69, 0x64, 0xa, 0x30, 0x50, 0x5a, 0x41, 0x69, 0x6e, 0x2b, 0x77, 0x37, 0x50, 0x53, 0x38, 0x6d, 0x54, 0x68, 0x63, 0x4a, 0x65, 0x42, 0x56, 0x4a, 0x73, 0x4d, 0x31, 0x6a, 0x34, 0x2b, 0x2f, 0x50, 0x36, 0x70, 0x42, 0x68, 0x63, 0x56, 0x2b, 0x61, 0x34, 0x66, 0x61, 0x64, 0x78, 0x56, 0x55, 0x34, 0x73, 0x58, 0x36, 0x43, 0x73, 0x67, 0x55, 0x4f, 0x58, 0x61, 0x4d, 0x32, 0x30, 0x58, 0x76, 0x63, 0x63, 0x33, 0x5a, 0xa, 0x76, 0x2b, 0x6b, 0x4f, 0x2f, 0x46, 0x5a, 0x63, 0x4c, 0x66, 0x79, 0x45, 0x34, 0x55, 0x4e, 0x30, 0x78, 0x37, 0x4a, 0x34, 0x6d, 0x63, 0x33, 0x32, 0x64, 0x7a, 0x33, 0x59, 0x43, 0x75, 0x75, 0x62, 0x79, 0x4a, 0x65, 0x6c, 0x6b, 0x75, 0x74, 0x50, 0x39, 0x41, 0x67, 0x66, 0x73, 0x63, 0x35, 0x4b, 0x4b, 0x77, 0x51, 0x48, 0x74, 0x52, 0x66, 0x63, 0x61, 0x75, 0x6d, 0x42, 0x71, 0x61, 0x50, 0x64, 0xa, 0x56, 0x73, 0x64, 0x74, 0x33, 0x7a, 0x33, 0x39, 0x6e, 0x77, 0x63, 0x6d, 0x32, 0x4d, 0x42, 0x74, 0x78, 0x41, 0x78, 0x6d, 0x56, 0x58, 0x42, 0x74, 0x64, 0x72, 0x70, 0x34, 0x4d, 0x44, 0x31, 0x37, 0x4d, 0x55, 0x73, 0x70, 0x77, 0x4b, 0x53, 0x4d, 0x58, 0x34, 0x58, 0x56, 0x38, 0x6b, 0x55, 0x61, 0x63, 0x50, 0x4d, 0x37, 0x55, 0x74, 0x6b, 0x33, 0x65, 0x31, 0x6b, 0x6a, 0x45, 0x77, 0x58, 0x74, 0xa, 0x6e, 0x6e, 0x52, 0x41, 0x34, 0x36, 0x6a, 0x4f, 0x76, 0x64, 0x4b, 0x64, 0x44, 0x54, 0x39, 0x30, 0x4f, 0x63, 0x32, 0x71, 0x71, 0x70, 0x52, 0x6a, 0x70, 0x56, 0x37, 0x68, 0x32, 0x4b, 0x61, 0x67, 0x73, 0x51, 0x49, 0x63, 0x57, 0x62, 0x45, 0x50, 0x49, 0x35, 0x43, 0x4c, 0x41, 0x73, 0x5a, 0x4a, 0x75, 0x45, 0x30, 0x58, 0x6d, 0x5a, 0x6c, 0x47, 0x2f, 0x68, 0x64, 0x34, 0x48, 0x68, 0x57, 0x31, 0xa, 0x70, 0x63, 0x67, 0x61, 0x35, 0x31, 0x67, 0x33, 0x62, 0x42, 0x49, 0x6d, 0x42, 0x78, 0x33, 0x30, 0x6a, 0x31, 0x4a, 0x64, 0x76, 0x51, 0x49, 0x44, 0x41, 0x51, 0x41, 0x42, 0x6f, 0x30, 0x49, 0x77, 0x51, 0x44, 0x41, 0x4f, 0x42, 0x67, 0x4e, 0x56, 0x48, 0x51, 0x38, 0x42, 0x41, 0x66, 0x38, 0x45, 0x42, 0x41, 0x4d, 0x43, 0x41, 0x71, 0x51, 0x77, 0x44, 0x77, 0x59, 0x44, 0x56, 0x52, 0x30, 0x54, 0xa, 0x41, 0x51, 0x48, 0x2f, 0x42, 0x41, 0x55, 0x77, 0x41, 0x77, 0x45, 0x42, 0x2f, 0x7a, 0x41, 0x64, 0x42, 0x67, 0x4e, 0x56, 0x48, 0x51, 0x34, 0x45, 0x46, 0x67, 0x51, 0x55, 0x41, 0x58, 0x6e, 0x72, 0x67, 0x69, 0x47, 0x73, 0x70, 0x39, 0x50, 0x36, 0x35, 0x37, 0x50, 0x31, 0x66, 0x38, 0x76, 0x72, 0x68, 0x62, 0x7a, 0x34, 0x69, 0x33, 0x6f, 0x77, 0x44, 0x51, 0x59, 0x4a, 0x4b, 0x6f, 0x5a, 0x49, 0xa, 0x68, 0x76, 0x63, 0x4e, 0x41, 0x51, 0x45, 0x4c, 0x42, 0x51, 0x41, 0x44, 0x67, 0x67, 0x45, 0x42, 0x41, 0x48, 0x39, 0x6d, 0x6b, 0x5a, 0x59, 0x67, 0x59, 0x42, 0x70, 0x79, 0x79, 0x54, 0x35, 0x56, 0x69, 0x61, 0x6e, 0x6c, 0x74, 0x34, 0x55, 0x74, 0x72, 0x70, 0x55, 0x35, 0x37, 0x69, 0x71, 0x49, 0x54, 0x69, 0x63, 0x4d, 0x7a, 0x32, 0x6c, 0x61, 0x66, 0x66, 0x50, 0x30, 0x6b, 0x65, 0x5a, 0x52, 0xa, 0x68, 0x75, 0x76, 0x39, 0x53, 0x34, 0x6e, 0x70, 0x56, 0x32, 0x4d, 0x50, 0x39, 0x33, 0x62, 0x36, 0x61, 0x5a, 0x33, 0x4c, 0x70, 0x59, 0x62, 0x42, 0x74, 0x62, 0x2b, 0x66, 0x63, 0x42, 0x67, 0x79, 0x45, 0x6a, 0x62, 0x37, 0x6e, 0x4d, 0x36, 0x75, 0x6d, 0x53, 0x72, 0x31, 0x64, 0x6f, 0x75, 0x4a, 0x39, 0x31, 0x37, 0x59, 0x79, 0x4c, 0x53, 0x42, 0x5a, 0x33, 0x70, 0x77, 0x6d, 0x32, 0x64, 0x34, 0xa, 0x57, 0x44, 0x68, 0x6f, 0x53, 0x48, 0x47, 0x32, 0x33, 0x33, 0x38, 0x55, 0x52, 0x56, 0x50, 0x4c, 0x6d, 0x68, 0x39, 0x75, 0x6e, 0x4e, 0x63, 0x6d, 0x6f, 0x2b, 0x2b, 0x51, 0x43, 0x76, 0x6b, 0x2b, 0x51, 0x62, 0x4a, 0x4f, 0x4f, 0x74, 0x55, 0x51, 0x34, 0x33, 0x75, 0x38, 0x79, 0x50, 0x66, 0x70, 0x58, 0x66, 0x37, 0x35, 0x6c, 0x68, 0x64, 0x77, 0x41, 0x69, 0x70, 0x63, 0x79, 0x38, 0x38, 0x6a, 0xa, 0x73, 0x55, 0x45, 0x53, 0x32, 0x4b, 0x63, 0x33, 0x59, 0x6e, 0x4a, 0x7a, 0x76, 0x32, 0x4a, 0x46, 0x49, 0x65, 0x2b, 0x47, 0x2b, 0x46, 0x63, 0x56, 0x4d, 0x57, 0x32, 0x44, 0x76, 0x79, 0x47, 0x76, 0x77, 0x5a, 0x5a, 0x75, 0x49, 0x68, 0x48, 0x71, 0x6f, 0x42, 0x47, 0x6c, 0x49, 0x38, 0x36, 0x47, 0x4f, 0x67, 0x67, 0x4d, 0x41, 0x56, 0x2b, 0x43, 0x31, 0x6c, 0x6b, 0x46, 0x5a, 0x43, 0x6d, 0x54, 0xa, 0x72, 0x43, 0x50, 0x63, 0x31, 0x67, 0x6d, 0x6a, 0x69, 0x6a, 0x41, 0x38, 0x4b, 0x34, 0x63, 0x74, 0x6d, 0x71, 0x39, 0x59, 0x56, 0x6f, 0x71, 0x37, 0x4d, 0x64, 0x5a, 0x7a, 0x31, 0x42, 0x77, 0x55, 0x67, 0x66, 0x33, 0x79, 0x48, 0x42, 0x63, 0x68, 0x35, 0x73, 0x33, 0x30, 0x32, 0x51, 0x2f, 0x7a, 0x76, 0x42, 0x45, 0x65, 0x64, 0x33, 0x33, 0x50, 0x36, 0x47, 0x74, 0x48, 0x61, 0x30, 0x45, 0x72, 0xa, 0x59, 0x6a, 0x6f, 0x2b, 0x36, 0x68, 0x50, 0x66, 0x7a, 0x37, 0x52, 0x50, 0x2b, 0x39, 0x62, 0x77, 0x61, 0x2b, 0x2f, 0x50, 0x6d, 0x6d, 0x53, 0x35, 0x4f, 0x46, 0x72, 0x6e, 0x44, 0x51, 0x62, 0x75, 0x7a, 0x53, 0x62, 0x74, 0x75, 0x6c, 0x6b, 0x3d, 0xa, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0x45, 0x4e, 0x44, 0x20, 0x43, 0x45, 0x52, 0x54, 0x49, 0x46, 0x49, 0x43, 0x41, 0x54, 0x45, 0x2d, 0x2d, 0x2d, 0x2d, 0x2d, 0xa}, NextProtos:[]string(nil)}
2024-05-10T01:20:05 info start to upgrade master0
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0510 01:20:05.934748  118450 utils.go:69] The recommended value for "healthzBindAddress" in "KubeletConfiguration" is: 127.0.0.1; the provided value is: 0.0.0.0
[preflight] Running pre-flight checks.
[upgrade] Running cluster health checks
[upgrade/version] You have chosen to change the cluster version to "v1.28.9"
[upgrade/versions] Cluster version: v1.27.9
[upgrade/versions] kubeadm version: v1.28.9
[upgrade/prepull] Pulling images required for setting up a Kubernetes cluster
[upgrade/prepull] This might take a minute or two, depending on the speed of your internet connection
[upgrade/prepull] You can also perform this action in beforehand using 'kubeadm config images pull'
2024-05-10T01:25:05 error upgrade cluster failed
Error: signal: killed```

[yangxggo]

not only upgrade from 1.27.9 to 1.28.9, if your upgrade target is greater than v1.28.0, you will have the same failure.

[yangxggo]

/assign