helm-charts
helm-charts copied to clipboard
connection refused config-reload container
Describe the bug
I deployed Jenkins-3.11.4 on Kubernetes v1.23 (Rancher Desktop) via Helm v.3.8.0. The issue is it takes a long time to pod up and running and when I checked the config-reload
container logs I am getting the below error. I do not know whether it is an expected issue once the pod is up and running but I also facing this issue when installing the manual plugin or if the pod restart somehow which then will not be up and running for a long time.
What might be the root cause of this issue? Is there anything that I have to do?
$ kubectl get pods -n jenkins
NAME READY STATUS RESTARTS AGE
jenkins-0 1/2 Running 0 81s
NLAMS1M0862:~ uralsem$ kubectl logs -f jenkins-0 -n jenkins -c config-reload
[2022-02-10 15:29:19] Starting collector
[2022-02-10 15:29:19] No folder annotation was provided, defaulting to k8s-sidecar-target-directory
[2022-02-10 15:29:19] Selected resource type: ('configmap',)
[2022-02-10 15:29:19] Loading incluster config ...
[2022-02-10 15:29:19] Config for cluster api at 'https://10.43.0.1:443' loaded...
[2022-02-10 15:29:19] Unique filenames will not be enforced.
[2022-02-10 15:29:19] 5xx response content will not be enabled.
[2022-02-10 15:29:24] Working on ADDED configmap jenkins/jenkins-jenkins-jcasc-config
[2022-02-10 15:29:57] Received unknown exception: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /reload-configuration-as-code/?casc-reload-token=jenkins-0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x4007fb31f0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred
Version of Helm and Kubernetes
- Helm: v3.8.0
- Kubernetes:v1.23.3
Chart version
jenkins -3.11.4
What happened?
1.helm upgrade --install jenkins . -n jenkins
2.kubectl logs -f jenkins-0 -n jenkins -c config-reload
What you expected to happen?
Jenkins pod up and running without any issue
How to reproduce it
helm upgrade --install jenkins . -n jenkins
Anything else we need to know?
Nope
That's not expected. It's ok if it fails on some tries. That's until Jenkins itself is started. But finally it should succeed.
If Jenkins startup takes too long you can adjust number of retries via https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/templates/jenkins-controller-statefulset.yaml#L321. Have a look at how that is configured via values https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/values.yaml#L354-L358
Check the documentation of the sidecar container for possible environment variables: https://github.com/kiwigrid/k8s-sidecar.
And obviously you could check why Jenkins startup takes so long. Maybe you need to adjust memory or CPU requests.
So as you can see below it restarts itself a bunch of times. Ok, let me check further.
$ kubectl get pods -n jenkins
NAME READY STATUS RESTARTS AGE
jenkins-0 2/2 Running 4 (42m ago) 55m
I am not talking about restarts of the pod. I meant retries within the kiwigrid sidecar. It's calling an http endpoint to trigger a config reload.
@torstenwalter I increased from 10 to 20 and also adjust cpu request. But it seems mostly a newtork issue where I do not have any idea what cause this.
Is the pod still restarting? Can you check logs of the jenkins
container?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.
I was having this problem when I configured SAML auth through Jcasc and forgot to add saml to installPlugins
.
I am facing same issue. pod keeps on restarting, below are the logs,
config-reload container logs:
[2022-05-28 18:30:26] Starting collector
[2022-05-28 18:30:26] No folder annotation was provided, defaulting to k8s-sidecar-target-directory
[2022-05-28 18:30:26] Selected resource type: ('configmap',)
[2022-05-28 18:30:26] Loading incluster config ...
[2022-05-28 18:30:26] Config for cluster api at 'https://172.20.0.1:443' loaded...
[2022-05-28 18:30:26] Unique filenames will not be enforced.
[2022-05-28 18:30:26] 5xx response content will not be enabled.
[2022-05-28 18:30:31] Working on ADDED configmap jenkins/jenkins-jenkins-jcasc-config
[2022-05-28 18:31:07] Received unknown exception: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /reload-configuration-as-code/?casc-reload-token=jenkins-0 (Caused by ResponseError('too many 500 error responses'))
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 846, in urlopen
return self.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 846, in urlopen
return self.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 846, in urlopen
return self.urlopen(
[Previous line repeated 2 more times]
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 836, in urlopen
retries = retries.increment(method, url, response=response, _pool=self)
File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /reload-configuration-as-code/?casc-reload-token=jenkins-0 (Caused by ResponseError('too many 500 error responses'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/resources.py", line 239, in _watch_resource_loop
_watch_resource_iterator(*args)
File "/app/resources.py", line 227, in _watch_resource_iterator
request(request_url, request_method, enable_5xx, request_payload)
File "/app/helpers.py", line 123, in request
res = r.post("%s" % url, auth=auth, json=payload, timeout=REQ_TIMEOUT)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 590, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 507, in send
raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /reload-configuration-as-code/?casc-reload-token=jenkins-0 (Caused by ResponseError('too many 500 error responses'))
jenkins container logs:
Running from: /usr/share/jenkins/jenkins.war
2022-05-28 18:34:24.860+0000 [id=1] INFO org.eclipse.jetty.util.log.Log#initialized: Logging initialized @523ms to org.eclipse.jetty.util.log.JavaUtilLog
2022-05-28 18:34:24.948+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file
2022-05-28 18:34:24.995+0000 [id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath
2022-05-28 18:34:25.052+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 11.0.15+10
2022-05-28 18:34:25.390+0000 [id=1] INFO o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
2022-05-28 18:34:25.421+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0
2022-05-28 18:34:25.421+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults
2022-05-28 18:34:25.422+0000 [id=1] INFO o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 600000ms
2022-05-28 18:34:25.940+0000 [id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
2022-05-28 18:34:26.117+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStart: Started w.@894858{Jenkins v2.332.3,/,file:///var/jenkins_cache/war/,AVAILABLE}{/var/jenkins_cache/war}
2022-05-28 18:34:26.136+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStart: Started ServerConnector@75d4a5c2{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2022-05-28 18:34:26.137+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: Started @1801ms
2022-05-28 18:34:26.169+0000 [id=23] INFO winstone.Logger#logInternal: Winstone Servlet Engine running: controlPort=disabled
2022-05-28 18:34:26.369+0000 [id=29] INFO jenkins.InitReactorRunner$1#onAttained: Started initialization
2022-05-28 18:34:26.599+0000 [id=31] INFO jenkins.InitReactorRunner$1#onAttained: Listed all plugins
2022-05-28 18:34:26.671+0000 [id=29] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Jackson 2 API Plugin v2.13.3-285.vc03c0256d517 (jackson2-api)
java.io.IOException: Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
- Update required: JAXB plugin (jaxb 2.3.0) to be updated to 2.3.6-1 or higher
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.672+0000 [id=29] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin ECharts API Plugin v5.3.2-2 (echarts-api)
java.io.IOException: Failed to load: ECharts API Plugin (echarts-api 5.3.2-2)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.672+0000 [id=29] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin JUnit Plugin v1.63 (junit)
java.io.IOException: Failed to load: JUnit Plugin (junit 1.63)
- Failed to load: ECharts API Plugin (echarts-api 5.3.2-2)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.673+0000 [id=29] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Matrix Project Plugin v771.v574584b_39e60 (matrix-project)
java.io.IOException: Failed to load: Matrix Project Plugin (matrix-project 771.v574584b_39e60)
- Failed to load: JUnit Plugin (junit 1.63)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.976+0000 [id=30] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline: REST API Plugin v2.24 (pipeline-rest-api)
java.io.IOException: Failed to load: Pipeline: REST API Plugin (pipeline-rest-api 2.24)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.977+0000 [id=31] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline: Stage View Plugin v2.24 (pipeline-stage-view)
java.io.IOException: Failed to load: Pipeline: Stage View Plugin (pipeline-stage-view 2.24)
- Failed to load: Pipeline: REST API Plugin (pipeline-rest-api 2.24)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.978+0000 [id=31] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Lockable Resources plugin v2.15 (lockable-resources)
java.io.IOException: Failed to load: Lockable Resources plugin (lockable-resources 2.15)
- Failed to load: Matrix Project Plugin (matrix-project 771.v574584b_39e60)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.982+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline: Model API v2.2086.v12b_420f036e5 (pipeline-model-api)
java.io.IOException: Failed to load: Pipeline: Model API (pipeline-model-api 2.2086.v12b_420f036e5)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.983+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline: Declarative Extension Points API v2.2086.v12b_420f036e5 (pipeline-model-extensions)
java.io.IOException: Failed to load: Pipeline: Declarative Extension Points API (pipeline-model-extensions 2.2086.v12b_420f036e5)
- Failed to load: Pipeline: Model API (pipeline-model-api 2.2086.v12b_420f036e5)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.983+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline: Declarative v2.2086.v12b_420f036e5 (pipeline-model-definition)
java.io.IOException: Failed to load: Pipeline: Declarative (pipeline-model-definition 2.2086.v12b_420f036e5)
- Failed to load: Pipeline: Model API (pipeline-model-api 2.2086.v12b_420f036e5)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.984+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline v2.6 (workflow-aggregator)
java.io.IOException: Failed to load: Pipeline (workflow-aggregator 2.6)
- Failed to load: Lockable Resources plugin (lockable-resources 2.15)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.985+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Kubernetes Client API Plugin v5.12.2-193.v26a_6078f65a_9 (kubernetes-client-api)
java.io.IOException: Failed to load: Kubernetes Client API Plugin (kubernetes-client-api 5.12.2-193.v26a_6078f65a_9)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.986+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Metrics Plugin v4.1.6.2 (metrics)
java.io.IOException: Failed to load: Metrics Plugin (metrics 4.1.6.2)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.993+0000 [id=29] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Kubernetes Credentials Plugin v0.9.0 (kubernetes-credentials)
java.io.IOException: Failed to load: Kubernetes Credentials Plugin (kubernetes-credentials 0.9.0)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.994+0000 [id=30] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Kubernetes plugin v1.31.3 (kubernetes)
java.io.IOException: Failed to load: Kubernetes plugin (kubernetes 1.31.3)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.995+0000 [id=30] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin GitHub API Plugin v1.303-400.v35c2d8258028 (github-api)
java.io.IOException: Failed to load: GitHub API Plugin (github-api 1.303-400.v35c2d8258028)
- Failed to load: Jackson 2 API Plugin (jackson2-api 2.13.3-285.vc03c0256d517)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.995+0000 [id=30] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin GitHub plugin v1.34.3 (github)
java.io.IOException: Failed to load: GitHub plugin (github 1.34.3)
- Failed to load: GitHub API Plugin (github-api 1.303-400.v35c2d8258028)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:26.996+0000 [id=30] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline: Declarative Agent API v1.1.1 (pipeline-model-declarative-agent)
java.io.IOException: Failed to load: Pipeline: Declarative Agent API (pipeline-model-declarative-agent 1.1.1)
- Failed to load: Pipeline: Declarative Extension Points API (pipeline-model-extensions 2.2086.v12b_420f036e5)
at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1018)
at hudson.PluginManager$2$1$1.run(PluginManager.java:542)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:29.099+0000 [id=29] INFO jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
2022-05-28 18:34:29.125+0000 [id=31] INFO jenkins.InitReactorRunner$1#onAttained: Started all plugins
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/var/jenkins_cache/war/WEB-INF/lib/groovy-all-2.4.21.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2022-05-28 18:34:29.208+0000 [id=31] INFO jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
2022-05-28 18:34:30.201+0000 [id=28] INFO jenkins.InitReactorRunner$1#onAttained: System config loaded
2022-05-28 18:34:30.535+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed ConfigurationAsCode.init
java.lang.IllegalArgumentException: No hudson.slaves.Cloud implementation found for kubernetes
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lambda$lookupDescriptor$11(HeteroDescribableConfigurator.java:211)
at io.vavr.control.Option.orElse(Option.java:321)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lookupDescriptor(HeteroDescribableConfigurator.java:210)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lambda$configure$3(HeteroDescribableConfigurator.java:84)
at io.vavr.Tuple2.apply(Tuple2.java:238)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.configure(HeteroDescribableConfigurator.java:83)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.check(HeteroDescribableConfigurator.java:92)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.check(HeteroDescribableConfigurator.java:55)
at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:343)
at io.jenkins.plugins.casc.BaseConfigurator.check(BaseConfigurator.java:286)
at io.jenkins.plugins.casc.ConfigurationAsCode.lambda$checkWith$8(ConfigurationAsCode.java:776)
at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:712)
at io.jenkins.plugins.casc.ConfigurationAsCode.checkWith(ConfigurationAsCode.java:776)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:761)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:637)
at io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:306)
at io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:298)
Caused: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:109)
Caused: java.lang.Error
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:115)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:185)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-28 18:34:30.537+0000 [id=22] SEVERE hudson.util.BootFailure#publish: Failed to initialize Jenkins
java.lang.IllegalArgumentException: No hudson.slaves.Cloud implementation found for kubernetes
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lambda$lookupDescriptor$11(HeteroDescribableConfigurator.java:211)
at io.vavr.control.Option.orElse(Option.java:321)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lookupDescriptor(HeteroDescribableConfigurator.java:210)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lambda$configure$3(HeteroDescribableConfigurator.java:84)
at io.vavr.Tuple2.apply(Tuple2.java:238)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.configure(HeteroDescribableConfigurator.java:83)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.check(HeteroDescribableConfigurator.java:92)
at io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.check(HeteroDescribableConfigurator.java:55)
at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:343)
at io.jenkins.plugins.casc.BaseConfigurator.check(BaseConfigurator.java:286)
at io.jenkins.plugins.casc.ConfigurationAsCode.lambda$checkWith$8(ConfigurationAsCode.java:776)
at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:712)
at io.jenkins.plugins.casc.ConfigurationAsCode.checkWith(ConfigurationAsCode.java:776)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:761)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:637)
at io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:306)
at io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:298)
Caused: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:109)
Caused: java.lang.Error
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:115)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:185)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused: org.jvnet.hudson.reactor.ReactorException
at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:291)
at jenkins.InitReactorRunner.run(InitReactorRunner.java:49)
at jenkins.model.Jenkins.executeReactor(Jenkins.java:1191)
at jenkins.model.Jenkins.<init>(Jenkins.java:981)
at hudson.model.Hudson.<init>(Hudson.java:86)
at hudson.model.Hudson.<init>(Hudson.java:82)
at hudson.WebAppMain$3.run(WebAppMain.java:247)
Caused: hudson.util.HudsonFailedToLoad
at hudson.WebAppMain$3.run(WebAppMain.java:264)
2022-05-28 18:34:30.548+0000 [id=22] INFO hudson.lifecycle.Lifecycle#onStatusUpdate: Stopping Jenkins
2022-05-28 18:34:30.571+0000 [id=22] INFO jenkins.model.Jenkins$16#onAttained: Started termination
2022-05-28 18:34:30.584+0000 [id=22] INFO jenkins.model.Jenkins$16#onAttained: Completed termination
2022-05-28 18:34:30.585+0000 [id=22] INFO jenkins.model.Jenkins#_cleanUpDisconnectComputers: Starting node disconnection
2022-05-28 18:34:30.587+0000 [id=22] INFO jenkins.model.Jenkins#_cleanUpShutdownPluginManager: Stopping plugin manager
2022-05-28 18:34:30.593+0000 [id=22] INFO jenkins.model.Jenkins#_cleanUpPersistQueue: Persisting build queue
2022-05-28 18:34:30.598+0000 [id=22] INFO jenkins.model.Jenkins#_cleanUpAwaitDisconnects: Waiting for node disconnection completion
2022-05-28 18:34:30.598+0000 [id=22] INFO hudson.lifecycle.Lifecycle#onStatusUpdate: Jenkins stopped
aniket@LAPTOP-N4HENFTO:~$ helm list -n jenkins
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
jenkins jenkins 1 2022-05-07 12:28:44.2928851 +0530 IST deployed jenkins-3.12.2 2.332.3
Do we have any solution on this?
Take a look at the error message, you need to update a plugin
same issue is face with me as well that jaxb plugin is failed on our statefulset jenkins deployed on GCP due to which cannot figure out how update the plugin and the pod is destroyed at every startup so UI is also not accessible.
Can anyone help here as how to update plugin on statefulset configuration on GCP.
if your plugins are managed as code just delete the /var/jenkins_home/plugins
folder and it should sort itself on startup
if your plugins are managed as code just delete the
/var/jenkins_home/plugins
folder and it should sort itself on startup
No plugins are managed by statefulset configmap file and cannot delete complete file just need to add these plugins explicitly to the file can you please suggest about the process as i am a newbie to jenkins and whole process to GCP.