antrea icon indicating copy to clipboard operation
antrea copied to clipboard

kubelet service stops working after restart on Windows Node in CI tesbed

Open wenyingd opened this issue 11 months ago • 0 comments

Describe the bug

After restart kubelet service on a Windows Node deployed from CI env, the service is in "paused" status.

PS C:\cygwin\home\Administrator> Restart-Service kubelet
PS C:\cygwin\home\Administrator> Get-Service kubelet 

Status   Name               DisplayName
------   ----               -----------
Paused   kubelet            kubelet

Trying to manually run kubelet commands, I got this error,

PS C:\k> .\StartKubelet.ps1
Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --pod-infra-container-image has been deprecated, will be removed in a future release. Image garbage collector will get sandbox image information from CRI.
Flag --pod-infra-container-image has been deprecated, will be removed in a future release. Image garbage collector will get sandbox image information from CRI.
Flag --enable-debugging-handlers has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --cgroups-per-qos has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --enforce-node-allocatable has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0314 02:39:45.864858    7872 server.go:203] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
I0314 02:39:45.884923    7872 server_windows.go:40] "Kubelet is running as" login name="A-MS-0008-WIN-1\\Administrator" dispaly name=""
I0314 02:39:45.885482    7872 server.go:467] "Kubelet version" kubeletVersion="v1.28.0"
I0314 02:39:45.886084    7872 server.go:469] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0314 02:39:45.886634    7872 init_windows.go:82] "Creating a Windows job object and adding kubelet process to it" windowsPriorityClass="NORMAL_PRIORITY_CLASS"
I0314 02:39:45.888951    7872 server.go:895] "Client rotation is on, will bootstrap in background"
E0314 02:39:45.890583    7872 bootstrap.go:241] unable to read existing bootstrap client config from /etc/kubernetes/kubelet.conf: invalid configuration: [unable to read client-cert C:\etc\kubernetes\var\lib\kubelet\pki\kubelet-client-current.pem for default-auth due to open C:\etc\kubernetes\var\lib\
elet-client-current.pem for default-auth due to open C:\etc\kubernetes\var\lib\kubelet\pki\kubelet-client-current.pem: The system cannot find the path 
E0314 02:39:45.892318    7872 run.go:74] "command failed" err="failed to run Kubelet: unable to load bootstrap kubeconfig: CreateFile /etc/kubernetes/bootstrap-kubelet.conf: The system cannot find the file specified."
E0314 02:43:52.158030    3076 run.go:74] "command failed" err="unknown command version"

It looks some references configured kubelet command parameter doesn't exist, we may need to refine the configurations in Prepare-Node.ps1

To Reproduce

  1. Logon to a Windows Node deployed in CI env
  2. Check the current kubelet service status with powershell command get-service kubelet
  3. Try to restart kubelet service restart-service kubelet
  4. Check the new status of kubelet service

Expected

kubelet service is supposed to be in "Running" status after restart.

Actual behavior

kubelet is paused.

Versions:

kubernetes v1.28.0

PS C:\k> .\kubelet.exe --version
Kubernetes v1.28.0

Additional context

wenyingd avatar Mar 15 '24 02:03 wenyingd