Groovy Expression in Free-Flow jobs is either broken or not documented enough to be useable.

[f-steff]

Originally documented here - probably with better formatting than here

Now added to Jenkins JIRA as [JENKINS-44389|https://issues.jenkins-ci.org/browse/JENKINS-44389]

I was initially attempting to get the Lockable Resources Plugin to lock on a specific resource while also evaluation the value of a system environment variable. Evaluation the system environment variable is working perfectly, however, I can't seem to get a lock on the correct resource. This question focused on the specific locking problem! I have created a three resources called A_TEST, B_TEST and C_TEST. None of them have any Labels. They are all visible from my Jenkins_URL/lockable-resources/ where they can be taken and released without problems. In my Jenkins Job configuration, I have selected the This build requires lockable resources option, which allows me to specify a Resource, Label or Groovy Expression (and Additional classpath). It additionally allows me to specify Reserved resources variable name and Number of resources to request. According to the documentation, the Groovy script must return a boolean value, so I determined to try it out:

Test 1 The first test I did was to verify the basic functionality, by setting the following: Resource = B_TEST Groovy Expression = Not checked Number of resources to request = 1 This results in the job executing with a lock on the B_TEST resource. The console output reads: [lockable-resources] acquired lock on [B_TEST]

Test 2 In this test I set the following: Resource = B_TEST Groovy Expression = Checked Groovy Script = return false Number of resources to request = 1 When attempting to execute the job, this wrongly waits forever with the text: (pending--Waiting for the resourced [B_TEST])

Test 3 In this test I set the following: Resource = B_TEST Groovy Expression = Checked Groovy Script = return true Number of resources to request = 1 This results in the wrong resource A_TEST to be locked. The console output reads: [lockable-resources] acquired lock on [A_TEST]

Test 4 After rereading the help for each option in the plugin, I discovered that the plugin apparently only allows me to specify either a Resource, Label or Groovy Expression So in this test I set the following: Groovy Expression = Checked Groovy Script = return false Reserved resources variable name = MyResourceName This results in the job executing wrongly without a lock on any resource. The console output reads: [lockable-resources] acquired lock on []

Test 5 So in this test I set the following: Groovy Expression = Checked Groovy Script = return true Reserved resources variable name = MyResourceName This results in the job wrongly locking on all resource. The console output reads: [lockable-resources] acquired lock on [A_TEST, B_TEST, C_TEST]

Test 6 According to the documentation of the plugin, in Jenkins-Pipelines this plugin can be used as this: echo 'Starting' lock('my-resource-name') { echo 'Do something here that requires unique access to the resource' // any other build will wait until the one locking the resource leaves this block } echo 'Finish' so I started experimenting with the Groovy Script containing variations of the lock('B_TEST') call, but this mostly lead to more confusion and errors while the job attempted to start, such as this: No signature of method: Script1.lock() is applicable for argument types: (java.util.LinkedHashMap) values: [[resource:B_TEST]] Possible solutions: each(groovy.lang.Closure), wait(), run(), run(), any(), find()) But I guess this all makes good sense, as the lock(){ } call seems suited to take and release the lock only during its scope.

The Question The big question now is, how does all of this actually work? My guess is that somewhere there's a groovy command to specify the Resources/Labels you want to reserve, and the return value (true/false) determines to actually take the lock or not.

[jimklimov]

Hi, looking at this today for my setup - it seems that when the Lockable Resources plugin sees a queued job with enabled groovy script, it calls the script for each single definition of a lockable resource, independently. It is up to your script to match the strings in resourceName and resourceLabel, maybe resourceDescription too.

The groovy script ultimately returns true or false, so the plugin has an array of accepted resourceName's - and then picks one (assuming you've required "1" as the amount in settings) to lock and return into the build environment in the variable name you've set up for the job. If none of the picked resources are currently available, the job stays in queue and the loop repeats after a few seconds.

Thanks to your write-up, as well as https://github.com/jenkinsci/lockable-resources-plugin/blob/master/src/main/resources/org/jenkins/plugins/lockableresources/RequiredResourcesProperty/help-resourceMatchScript.html for helping figure this out. The println routine and jenkins.log were also very helpful ;)

Now, one more quest for me remains - I want this logic to interact with build arguments of the queued job, and those seem to not be available in the context of this groovy script :( UPDATE: Hopefully PR #72 will address this issue of mine.

[f-steff]

Hi @jimklimov, Thank you for your investigations and findings. Not least the comments on issue #73 which seems really interesting.

I'll see if I can find time to work in it this week, and if successful will update the Jenkins Wiki with simple examples.

If you have discovered more valuable info, please let me know.

Thanks.

[f-steff]

@jimklimov To access the build-parameters passed to the job, you can use: import hudson.model.* def JobParameter = System.getenv("YOUR_PARAMETER_PASSED_TO_THE_SCRIPT")

I tested your hypothesis, by dumping all variables available to the Groovy script: println "Lockable Resource Groovy Script Start" this.binding.variables.each {k,v -> println "$k = $v"} println "Lockable Resource Groovy Script End" println "" return false

This resulted in the following output repeated over and over when the job attempted to start:

Lockable Lockable Resource Groovy Script Start resourceLabels = [Node2A, Node2B, Node2C] resourceDescription = This is a TEST resource. resourceName = Node2 Lockable Resource Groovy Script End

Lockable Lockable Resource Groovy Script Start resourceLabels = [Node1A, Node1B, Node1C] resourceDescription = This is a TEST resource. resourceName = Node1 Lockable Resource Groovy Script End

So it's possible to match resourceName and resourceLabels, and it's possible to return true or false.

If true is returned, all Names and Labels are locked - unless the Number of resources to request value is not a positive number, in which case the job runs without checking the resources.

So effectively it's possible to use this as a simple semaphore, based on the resourceName, since using resourceLabels doesn't add any value.

It would be nice if it was possible to return true/false to select on the resourceName level or a resourceLabels list to specify the labels this jobs selected to have locked.

[jimklimov]

Note that by now my PR is merged. When I was doing similar experiments, I did not see build args as envvars in the evaluated lockable-resources groovy script. It is also quite possible I botched the experiment somehow, not being proficient in Java ;) But I did find "teh codez" on the net to print out all envvars seen by the script's context - and build args were not there for me.

I am not sure I got all your further elaboration about resource matching, but note that locking regards each individual resourceName which often marks some limited (and unshareable) physical resource. It is quite valid for different instances to have same labels (maybe part of different larger label sets) so you can declare "give me any one of these, exclusively while this job runs"!

[f-steff]

Hi again @jimklimov, Sorry for my late delay.

It's great that your PR is merged, but I wonder when it will be released in the plugin?

About my further elaborations, it was based on my understanding of the plugin, which again was based on the old online (and in plugin) help text. As far as I remember, it stated that one resource (name) could have several "sub-resources" (labels), and that you could choose to lock on the all (name), or a few specific, leaving the others available for other jobs.

To me it made sense like this: Resource name: Transmitters Resource labels: Transmitter1, Transmitter2, Transmitter3, Transmitter4 #of resources requested: n

So with the above setup, I could have two ways to lock resources.

1. Specify "Transmitters" as Resource name: I would end up getting the amount requested (n), out of the four available transmitters in the pool. If I requested 2, another job could get the other two.

2. Specify the labels to get some specific resources out of the pool.

As I understand the implementation as is is now, the usage of name's or label's doesn't make any difference - as it locks the same resource. And I even less understand the usage of the the amount of resources to lock... - dont you always get all of them?

[TobiX]

@f-steff I think you confused resources and labels. A resource is always the one thing that is locked, it exists once and has an unique name (if we take the hardware example, this may be office_printer_14). Every resource can have multiple labels (the printer could be labeled dot-matrix-printer, in-office-printer, a4-printer, etc.). All resources with the same label form a "pool", so if you try to lock 1x a4-printer, you might get a random resource which has the label a4-printer - if all resources with the label a4-printer are in use, your job waits until one is available.