jenkinsci
Reoccurring null in LockableResourcesManager resources
Jenkins and plugins versions report
Environment
Jenkins: 2.426.2
OS: Linux - 5.15.0-1064-azure
Java: 17.0.9 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
lockable-resources:1218.va_3dd45e2b_fa_7
What Operating System are you using (both controller, and any agents involved in the problem)?
Jenkins is running in a container on Ubuntu 22
Reproduction steps
Unfortunately I do not have exact steps to reproduce the issue. I have a number of jobs that use lockable resources, so resources are locked/unlocked continuously minimum few times a minute. I've observed that every few days somehow a null is added to LockableResourcesManager resources list. This breaks not only the plugin (any call to step lock throws exception), but also global Jenkins configuration and the only option to fix it, is to directly replace resources list in LockableResourcesManager.
I'm not using any advanced features of this plugin, only simple step lock('lock-name') { }
I was tryin to find any logs from when this null was added, but I didn't find any.
I doubt it matters, but in my case job name is often used as resource name.
Expected Results
I don't know how is that even possible that a null resource is added to the LockableResourcesManager.resources, but I would expect that LockableResourcesManager should be able to remove it as clearly invalid entry.
Actual Results
I've observed two logs in case of failure (but I'm not sure why there are two variants) - both often appear mixed in log of the same job, that attempts to lock always the same resource.
Trying to acquire lock on [<lock name>]
[Pipeline] // lock
[Pipeline] End of Pipeline
java.lang.NullPointerException
Finished: FAILURE
and longer one:
[Pipeline] lock
Trying to acquire lock on [<lock name>]
[Pipeline] // lock
[Pipeline] End of Pipeline
Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: a326e53e-1b63-49e1-9293-60b260f97d14
java.lang.NullPointerException: Cannot invoke "org.jenkins.plugins.lockableresources.LockableResource.getName()" because "r" is null
at org.jenkins.plugins.lockableresources.LockableResourcesManager.fromName(LockableResourcesManager.java:272)
at org.jenkins.plugins.lockableresources.LockableResourcesManager.resourceExist(LockableResourcesManager.java:292)
at org.jenkins.plugins.lockableresources.LockableResourcesManager.addResource(LockableResourcesManager.java:803)
at org.jenkins.plugins.lockableresources.LockableResourcesManager.createResource(LockableResourcesManager.java:753)
at org.jenkins.plugins.lockableresources.LockStepExecution.start(LockStepExecution.java:55)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:323)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:47)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
at org.jenkinsci.plugins.workflow.cps.LoggingInvoker.methodCall(LoggingInvoker.java:105)
at discardOtherConcurrent.call(discardOtherConcurrent.groovy:6)
at discardOtherConcurrent.call(discardOtherConcurrent.groovy)
at WorkflowScript.run(WorkflowScript:32)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:90)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:116)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:85)
at jdk.internal.reflect.GeneratedMethodAccessor247.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:152)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:146)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:97)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Finished: FAILURE
Anything else?
No response
Are you interested in contributing a fix?
No response
Same problem with Jenkins 2.426.2. Message ( java.lang.NullPointerException Cannot invoke "org.jenkins.plugins.lockableresources.LockableResource.getName()" because "r" is null) seemingly at the exit from steps.lock() pipeline block. This occurred for the second time in a few days. Before that the same Jenkins installation worked smoothly for several months. Recently have added some agent but it seems to work well...
Same problem with Jenkins 2.426.2. Message ( java.lang.NullPointerException Cannot invoke "org.jenkins.plugins.lockableresources.LockableResource.getName()" because "r" is null) seemingly at the exit from steps.lock() pipeline block. This occurred for the second time in a few days. Before that the same Jenkins installation worked smoothly for several months. Recently have added some agent but it seems to work well...
which plugin version do you used?
1218
may you pls update the plugin to the last one version. There was many fixes and this issue is porbably fixed.
thx
This plugin update probably requires total Jenkins update. Not so easy in our working environment. Will try when I can...
I believe the problem was this. There were three pipelines named say P1, P2, P3. The first and the third pipeline use resources (R1,...RN). The second pipeline uses resources (R2...RN) except for R1. Sequence of events could be like this:
- P1 starts and locks (R1..RN) and does its work.
- P2 starts and waits for (R2..RN).
- P1 finishes, releasing (R1..RN).
- P2 locks (R2..RN) and does its work. At this point R1 is presumably free or may even be destroyed.
- P3 starts and waits for (R1..RN).
- P2 completes and releases (R2..RN). At this moment an exception occures in P2 as described above in this issue.
The above model is my best guess. But the fact is that actually I was able to circumvent the problem by including R1 into the list of required resources for the P2 pipeline. So now when all the resources lock and release simultaneously it all works smoothly as before. That is my habit anyway. The problem initially arised some weeks ago when I forgot to include a new resource R1 into P2. Regards.
Nice scenario. I will propose to create an automated test first. Then we can see if the issue is still rproducable.