kubeblocks icon indicating copy to clipboard operation
kubeblocks copied to clipboard

[BUG] restart a component with evicted pod failed

Open nayutah opened this issue 5 months ago • 4 comments

Describe the bug A mysql cluster has a mysql component with two replicas, and one replica is evicted due to disk pressure of node ephemeral-storage. When the disk pressure recovers to normal (< 85%) state, the evicted Pod is still in ContainerStatusUnknown state, and we start a 'restart' ops to restart the mysql cluster, then the ops is stuck and two replicas are in ContainerStatusUnknown state.

To Reproduce Steps to reproduce the behavior:

  1. Create a mysql primary-secondary cluster
  2. Make the system disk with a disk pressure level more than 85%
  3. One of the mysql replica is evicted, and in state ContainerStatusUnknown
  4. Restart the mysql cluster
  5. Two replicas are both in ContainerStatusUnknown state

Expected behavior 1, Reject to restart for a failed or abnormal component 2, Or permit restarting the failed/abnormal component, and goes well

Screenshots

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context Add any other context about the problem here.

nayutah avatar Aug 27 '24 06:08 nayutah