allow to link a lockable resource to a node

[monger39]

What feature do you want to see added?

in our (mostly scripted) pipelines we need to use a single resource for which an agent can have between 0-4 of such resources. Not each agent has such resource, some have 2, some 3, some 4. Typically a single resource is locked by one of the 'parallel' (sub-)builds of a pipeline. Agents typically have 6 execution slots. The resources are identified by a generic label. The challenge for the pipeline job is to lock a free resource available through an online node where an execution slot is available.

In our current solution each resource has in its name the name of a node, and an index nr on that node; for example this could be "virtmachA_devres01", "virtmachA_devres02", .. where "virtmachA" is the name of an agent. Then, in the pipeline script, we request a lock with label "devres" (the label assigned to all the resources). Given the resource, we extract the agent name, and assign the particular code t that, using the "node(derived_agent) {...}". This mostly works, unless the node is offline, or all execution slots are taken. It is also annoying when the node has to be taken offline for maintenance; all resources for that agent need to be reserved manually.

My enhancement proposal is to extend the LockableResource with an (optional) 'agentname'. When defined:

• getting (locking) the resource can only be done when the agent is online and has a free slot
• the name of the agent is available through the resource.agent , thus allowing to schedule the workload on that agent.

This is maybe related to #341 and/or #309. However I think this proposal would need only a little work in the plugin, whereas it makes the pipeline scripts much easier and more robust.

Upstream changes

No response

[mPokornyETM]

This is exact the reason, why I created the #341 I think all of the magic, what you explain can be done in groovy as well. An helper in our shared-library will be great.

import org.jenkins.plugins.lockableresources.LockableResourcesManager as LRM

@NonCPS
def getResource(String name)
{
  def allr = LRM.get().getResources();
  for(int i = 0; i < allr.size(); i++)
  {
    if ( allr[i] && (allr[i].getName() == name) )
      return allr[i];
  }

  return null;
}

def filteredNodes = [];
int expectedCount = 4;
// the best way to synchronize  code between jobs is to use lockable-resources ;-)
lock('get-nodes') {
  jenkins.model.Jenkins.instance.computers.each { c ->
     if (filteredNodes.size() == expectedCount)
       return;

      if (c == null || c.node == null) {
        return; // sometimes (no idea why) c.node is null
      }

      if (!c.isOnline()) {
        // is offline ignore it
        return;
      }

      String nodeName = c.node.selfLabel.name;
      
     der resource = getResource(nodeName);

     if (resource == null) {
        // resources does not exist, create it
        return;
     }

    if (resource.isLocked() || resource.isQueued() || resource.isReserved()) {
      // occupated
      return;
    }

     filteredNodes.push(nodeName);;
  }
}

echo 'my free and online nodes: ' + filteredNodes

ps: I type it here just so, without syntax check or what ever, But I thing as idea it shall helps

[mPokornyETM]

I think I provide the functionality here https://github.com/mPokornyETM/jenkins-lockable-resources-shared-library/blob/feature/step/vars/lockNode.groovy @monger39 Has you time to check it if it will work for you?

[monger39]

Hi @mPokornyETM , I have tried understanding how the code would work to solve my problem, but unfortunately I don't see it ... Could you provide a hint how to use the library ? I would try to test, but the real test is in an air-gapped system so takes a bit of time to get the stuff setup. Especially because I do not use shared libs at all (yet / still)...

[mPokornyETM]

Yes of course. I will provide examples in the #457 . ! This is other location, because it will be much more easier to maintain it this way. I am next week on vacation. Therefore I can continue later here. Anyway I will write here when I am done.

[monger39]

mea culpa @mPokornyETM for not replying sooner. I was having (and still have) severe problems getting a test environment up. Just want to let you know that I have not yet forgotten and will test as soon as my systems is up again.

[mPokornyETM]

@monger39 is fine. We are in open sorece community, no presure here. When you has time it is welcome, when not it is accepted ;-)