installPlugins issues, git version behind other plugins, kubernets version unable to deploy agents

[SimonStiil]

Describe the bug

Multiple plugins have started requirering git:5.2.0. But the chart still selects git:5.1.0

There seems to be a bug in kubernetes:3937.vd7b_82db_e347b_ that preverts jenkins from starting agents in kubernetes with an error about empty namespace even though namespace for agents is defined in chart. This error is fixed in later versions of the plugin. Using kubernetes:4029.v5712230ccb_f8 it works without issues. Probably will work with earlier builds but have only tried the one.

Version of Helm and Kubernetes

- flux: v2.0.1
- Kubernetes: v1.25.9

Chart version

jenkins-4.6.1

What happened?

1. Upgraded helm chart to v1.25.9
2. Chart fetched new plugins
3. Chart unable to start due to pinned git:5.1.0
4. Tried setting git to git:5.2.0
5. Start a build that should start a agentpod.
6. agentpod would not start with a long log with fabric8 errors (dependency of kubernetes plugin)

What you expected to happen?

Upgrade chart. Jenkins starts. Jenkins able to build in agent pod.

How to reproduce it

controller:
      additionalPlugins:
      - ldap
      - http_request
      - pipeline-maven
      - pipeline-utility-steps
      - pipeline-stage-view
      - pipeline-github
      - github-api
      - github-branch-source
      - github-autostatus
      - job-dsl
      - basic-branch-build-strategies
      - email-ext
      - mailer
      - timestamper
      - kubernetes-credentials-provider
      - credentials-binding
      - generic-webhook-trigger
    persistence:
      storageClass: nfs-csi
    rbac:
      create: true
      readSecrets: true

Anything else we need to know?

no

[jseiser]

We have a prod/dev setup with jenkins, and they are deploying using the same helm chart + values except for the ingress name.

We randomly started getting this error, on a working system ( but only in dev )

2023-09-20 16:23:21.772+0000 [id=38]	INFO	hudson.slaves.NodeProvisioner#update: default-9tcrd provisioning successfully completed. We have now 2 computer(s)
2023-09-20 16:23:21.782+0000 [id=2317]	WARNING	o.c.j.p.k.KubernetesLauncher#launch: Error in provisioning; agent=KubernetesSlave name: default-9tcrd, template=PodTemplate{id='a6e410bbc54d23e84ac3f58bd1608da0e09b1ef712ce19292fc4c1ee6c8add69', name='default', namespace='jenkins', slaveConnectTimeout=100, label='jenkins-jenkins-agent', serviceAccount='jenkins-worker', nodeUsageMode=NORMAL, podRetention='Never', containers=[ContainerTemplate{name='jnlp', image='jenkins/inbound-agent:3107.v665000b_51092-15', workingDir='/home/jenkins/agent', args='${computer.jnlpmac} ${computer.name}', resourceRequestCpu='512m', resourceRequestMemory='512Mi', resourceLimitCpu='512m', resourceLimitMemory='512Mi', envVars=[KeyValueEnvVar [getValue()=http://jenkins.jenkins.svc.cluster.local:8080/, getKey()=JENKINS_URL]]}]}
java.lang.IncompatibleClassChangeError: Found class io.fabric8.kubernetes.api.model.VolumeFluent$EmptyDirNested, but interface was expected
	at org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume.buildVolume(EmptyDirWorkspaceVolume.java:63)
	at org.csanchez.jenkins.plugins.kubernetes.PodTemplateBuilder.build(PodTemplateBuilder.java:211)
	at org.csanchez.jenkins.plugins.kubernetes.PodTemplate.build(PodTemplate.java:902)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:119)
	at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:297)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
2023-09-20 16:23:21.782+0000 [id=2317]	INFO	o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance for agent default-9tcrd
2023-09-20 16:23:21.782+0000 [id=2317]	WARNING	o.c.j.p.k.KubernetesSlave#deleteSlavePod: Failed to delete pod for agent null/default-9tcrd: namespace cannot be null
io.fabric8.kubernetes.client.KubernetesClientException: namespace cannot be null
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.inNamespace(BaseOperation.java:245)
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.inNamespace(BaseOperation.java:92)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave.deleteSlavePod(KubernetesSlave.java:357)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave._terminate(KubernetesSlave.java:343)
	at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:88)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:259)
	at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:297)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
2023-09-20 16:23:21.788+0000 [id=2317]	INFO	o.c.j.p.k.KubernetesSlave#_terminate: Disconnected computer default-9tcrd

I assume this is the name space issue you are referring too

[ArtursTrubacs]

Also have a problem with Jenkins, Jenkins not build a job, jenkins job stands with error - Still waiting to schedule task Waiting for next available executor

podName-65504-h41ws-6-d4z19 provisioning successfully completed. We have now 1 computer(s)
Apr 04, 2024 11:40:41 AM WARNING org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
Error in provisioning; agent=KubernetesSlave name: podName-65504-h41ws-6-39f42, template=PodTemplate{id='63c841c4-7ab5-4c5f-bb54-19bb775a2d81', name='CRONJOBS_JUPYTER_jupyter-trip-planning-data_65504-h41ws-6krfr', namespace='jenkins', label='CRONJOBS_JUPYTER_jupyter-trip-planning-data_65504-h41ws', annotations=[PodAnnotation{key='buildUrl', value='http://jenkins.jenkins.svc.cluster.local:8080/job/CRONJOBS/job/JUPYTER/job/jupyter-trip-planning-data/65504/'}, PodAnnotation{key='runUrl', value='job/CRONJOBS/job/JUPYTER/job/jupyter-trip-planning-data/65504/'}]}
java.lang.IncompatibleClassChangeError: Found class io.fabric8.kubernetes.api.model.VolumeFluent$EmptyDirNested, but interface was expected
	at org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume.buildVolume(EmptyDirWorkspaceVolume.java:63)
	at org.csanchez.jenkins.plugins.kubernetes.PodTemplateBuilder.build(PodTemplateBuilder.java:211)
	at org.csanchez.jenkins.plugins.kubernetes.PodTemplate.build(PodTemplate.java:902)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:119)
	at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:298)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Apr 04, 2024 11:40:41 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent podName-65504-h41ws-6-39f42
Apr 04, 2024 11:40:41 AM WARNING org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePod
Failed to delete pod for agent null/podName-65504-h41ws-6-39f42: namespace cannot be null
io.fabric8.kubernetes.client.KubernetesClientException: namespace cannot be null
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.inNamespace(BaseOperation.java:245)
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.inNamespace(BaseOperation.java:92)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave.deleteSlavePod(KubernetesSlave.java:357)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave._terminate(KubernetesSlave.java:343)
	at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:88)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:259)
	at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:298)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Apr 04, 2024 11:40:41 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Disconnected computer podName-65504-h41ws-6-39f42
Apr 04, 2024 11:40:41 AM INFO hudson.slaves.NodeProvisioner update
podName-65504-h41ws-6-39f42 provisioning successfully completed. We have now 1 computer(s)
Apr 04, 2024 11:40:42 AM WARNING org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch

[kvanzuijlen]

@ArtursTrubacs Can you check your plugin versions/see if they're the latest versions?

[SimonStiil]

From my point of view this can be resolved. Running chart version: 5.0.13 And all plugins are up to date.