Expose `.spec.podManagementPolicy` to make the pod rollback-able from the broken state

[UkonnRa]

Is your feature request related to a problem? Please describe

If the existing Pod is not in a healthy state and podManagementPolicies is OrderedReady(default), the controller won't create a new Pod to replace the old one. More precisely, the controller cannot even support statefulset update when the existing pod is stucking in broken state. The statefulset very often chooses to stop and do nothing if anything isn't going perfectly according to plan.

Related bug: https://github.com/kubernetes/kubernetes/issues/67250

Describe the solution you'd like

Expose the Pod Management Policies in values.yaml so when customers have that kind of requirements, they can change it to Parallel Mode to make the rollback feature workable.

Describe alternatives you've considered

No alternatives.

Additional context

No response

[torstenwalter]

I am not sure if Parallel will help there:

This option only affects the behavior for scaling operations. Updates are not affected.

https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#parallel-pod-management

Jenkins only supports running a single replica. So having more than one replica running will produce unpredictable results.

[flying-snail0630]

I hit a similar issue. The k8s controller cannot patch the Pod when Jenkins failed to start. I have to uninstall the Jenkins and reinstall it with the correct configuration. Directly running helm upgrade can change the manifest of StatefulSet, however, it won't affect the Pod as StatefulSet cannot update Pod when it is stuck in a broken state.

ENV

• helm: v3.8.0
• Kubernetes: v1.21.x
• replica: 1
• podManagementPolicies: OrderedReady

I also tested pod management policies as Parallel and replica=1. The change of spec template of StatefulSet would affect pod this time and it's actually an in-place update. So I don't need to uninstall/re-install the Jenkins chart. helm upgrade -i would always work.

[UkonnRa]

I hit a similar issue. The k8s controller cannot patch the Pod when Jenkins failed to start. I have to uninstall the Jenkins and reinstall it with the correct configuration. Directly running helm upgrade can change the manifest of StatefulSet, however, it won't affect the Pod as StatefulSet cannot update Pod when it is stuck in a broken state.

ENV

• helm: v3.8.0
• Kubernetes: v1.21.x
• replica: 1
• podManagementPolicies: OrderedReady

I also tested pod management policies as Parallel and replica=1. The change of spec template of StatefulSet would affect pod this time and it's actually an in-place update. So I don't need to uninstall/re-install the Jenkins chart. helm upgrade -i would always work.

Exactly, I tested on my cluster as well and Jenkins could be restarted smoothly as expected, rather than being stucked. Hoping it can be fixed.

[torstenwalter]

Anyone up to creating a PR for it?

[bsloan-icl]

Experienced the same issue. Would be great to expose the podManagementPolicy in the values file so it can be set to parallel instead which resolves the issue.