reset initialization-completed (controller.initializeOnce=true) iff Jenkins version != previous initialized version

[ohauer]

Is your feature request related to a problem? Please describe

We run a small number of Jenkins installtions on K8s, on this installations we have set "controller.initializeOnce: true" to keep plugins stable during restarts (e.g. move from worker node A to worker node B). With this setting we have to remove the file "{{ .Values.controller.jenkinsHome }}/initialization-completed" manually before each version update else we do not get fresh plugins.

Describe the solution you'd like

It would be nice to have something like the following snippet in "templates/config.yaml", this way the initialization-completed file is removed if jenkins version has changed.

--- jenkins/templates/config.yaml.orig
+++ jenkins/templates/config.yaml
@@ -14,6 +14,10 @@
   apply_config.sh: |-
     set -e
 {{- if .Values.controller.initializeOnce }}
+    if [ -f {{ .Values.controller.jenkinsHome }}/initialization-completed -a -e {{ .Values.controller.jenkinsHome }}/jenkins.install.UpgradeWizard.state ]; then
+      JENKINS_OLD_VER=$( awk '{print $1}' {{ .Values.controller.jenkinsHome }}/jenkins.install.UpgradeWizard.state )
+      [ "${JENKINS_OLD_VER}" = "${JENKINS_VERSION}" ] || rm -f {{ .Values.controller.jenkinsHome }}/initialization-completed
+    fi
     if [ -f {{ .Values.controller.jenkinsHome }}/initialization-completed ]; then
       echo "controller was previously initialized, refusing to re-initialize"
       exit 0

Describe alternatives you've considered

Similar to what we use today:

kubectl get po -A | awk '/jenkins-controller/ {printf "kubectl -n %s exec -it %s -c jenkins -- rm -f  /var/jenkins_home/initialization-completed\n", $1, $2}' | sh

Additional context

The snippet is just for discussion, if there are other already existing solutions I'm happy to adopt / test. Our target is to do as less as possible with kubectl as we roll out all jenkins from umbrella charts with ArgoCD.

[torstenwalter]

The typical solution to keep plugins stable is to bundle them in a custom image.

[ohauer]

But building 25 different different images for 60 jenkins is not really praktical, and installing all sort of plugins into every jenkins even not. We reduced installation for all jenkins by using an umbrella chart with common configuration and only project specific overwrites (e.g. URL, ingress, addtitionalPlugins, ...). Plugins are served via an internal proxy and the list of plugins is build once a day pointing to the internal plugin proxy. All in all this runs fine with the exception of this ticket

[timja]

Probably okay behind a flag? but yeah generally not recommended to do this because your Jenkins may fail to start if it can't download the plugins.

In your case this may be okay but generally not a great idea