k8s-spot-rescheduler
k8s-spot-rescheduler copied to clipboard
Is the moving of pods logged?
Because all I see is this:
I0502 14:44:16.294689 1 rescheduler.go:247] No nodes to process.
I0502 14:44:26.443325 1 rescheduler.go:247] No nodes to process.
I0502 14:44:36.574006 1 rescheduler.go:247] No nodes to process.
I0502 14:44:46.724541 1 rescheduler.go:247] No nodes to process.
I0502 14:44:56.940652 1 rescheduler.go:247] No nodes to process.
I0502 14:45:07.118794 1 rescheduler.go:247] No nodes to process.
I0502 14:45:17.301454 1 rescheduler.go:247] No nodes to process.
I0502 14:45:27.464235 1 rescheduler.go:247] No nodes to process.
And sometimes this:
I0502 14:46:28.288639 1 rescheduler.go:205] Waiting for unschedulable pods to be scheduled.
@max-rocket-internet did you get to the bottom of this?
It's not logged but it should definitely be.
Hey @max-rocket-internet and @sercanacar, which log setting do you have? The rescheduler uses glog and we typically run with level 2 -v=2
at @pusher. You are right that there is not specific log line for each individual pod being evicted, though i don't see why we couldn't add one, probably somewhere within this method https://github.com/pusher/k8s-spot-rescheduler/blob/b46cd362a3fb07159226b8d5a9872ad6f0de8dab/scaler/scaler.go#L71
which log setting do you have?
I'm just using the chart: https://github.com/helm/charts/tree/master/stable/k8s-spot-termination-handler
there is not specific log line for each individual pod being evicted
I would say this is very important to log at default log level. Much more important than No nodes to process.
😄
Yeah you're probably right... Levelled logging decisions are hard 😅
Any update on this please?