Unable to test pipelines using Global Libraries

[joecarlyon]

I have many pipelines that include a global library @Library('[email protected]') We wrap some usage, such as Docker support. The pipelines use that in their code. In the unit test, I am unable to overwrite the variables that get used in the library.

Pipeline Code:

def unitTestStep(data) {
  node {
    dockerSupport.insideContainer('docker.artifactory.company.com/company/alpine-docker-mvn-ci:0.1.0') {
    echo 'Running with Maven'
    sh "mvn test -P unit-tests -Dparam=${data}"```
  }
}

Unit Test Code:

void docker_example() {
    binding.setVariable("buildConfiguration", "Sample Build Configuration")
    def script = loadScript("path/to/pipelineCode.groovy")
    script.unitTestStep("This is my data. There are many like it, but this one is mine.")
    printCallStack()
}

Inside of dockerSupport.insideContainer there is code being called that uses buildConfiguration that is not getting set. The line binding.setVariable("buildConfiguration", "Sample Build Configuration") does not override this.

I'm unsure how to mock these things out to move forward.

[ozangunalp]

Just to be sure, you did registered your library to the framework as shown in https://github.com/lesfurets/JenkinsPipelineUnit#testing-shared-libraries right?

[joecarlyon]

Yeah, I checked that out and it's correct. I may have formed this question wrong, conflating two separate issues I'm facing. This is not my forte.

Global libraries load fine. I'm unable to find the right way to create a typed global variable form an installed plugin https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/docker/workflow/Docker.groovy (which gets added in https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/DockerDSL.java )

Also, how do we pass the org.jenkinsci.plugins.workflow.cps.CpsScript script (this in a Jenkinsfile) around?"

[ozangunalp]

I recommend you to take a look here : https://jenkins.io/doc/book/pipeline/shared-libraries/#writing-libraries

To resume, In your global library you may have classes in src directory that you can instantiate in your scripts. These classes don't have access to the script (this in your Jenkinsfile) unless you pass it as parameter (or in constructor). You can define a global variable in vars, which is instantiated and injected to your pipeline scripts as variables. In these variables you have access to any step (git, sh, etc.) and var (scm, currentBuild, etc.). So the buildConfiguration variable that you want to mock should be also available inside global variables.

Also note that loading libraries dynamically with library step is not yet supported by the testing framework.

[mkobit]

Hey @ozangunalp thanks for looking into this - hopefully I can explain what we think the current issue is.

We have a global library internally that has a bunch of classes that wrap a bunch of other steps. We essentially don't use anything in vars for a set of best practices that we have developed. Most of our code is written like normal "Groovy" in the src directory like the following:

In src/com/company/jenkins/DockerSupport.groovy we have code that wraps some of the steps in Jenkins pipelines. This is a contrived example, but will hopefully let me elaborate the current issue some more:

package com.company.jenkins

class DockerSupport implements Serializable {

  private final script

  /**
   * This constructor sets the Jenkins 'script' class as the local script
   * variable in order to resolve execution steps.(sh, withCredentials, etc)
   * @param script the script object
   */
  DockerSupport(script) {
    this.script = script
  }

  def insideContainer(String imageName, Closure body) {

    body.delegate = script
    body.resolveStrategy = Closure.DELEGATE_FIRST
    script.echo('Doing some work now')
    script.sh(returnStdout: true, script: 'docker ps').trim()
    script.node {
      script.docker.image(imageName).inside {
        body()
      }
    }
  }
}

And then, when one of our users wants to consume these libraries in their Jenkinsfile, they will do something like:

#!groovy
@Library('[email protected]')
import com.company.jenkins.DockerSupport

DockerSupport dockerSupport = DockerSupport(this)

dockerSupport.insideContainer('alpine:3.5') {
  sh 'echo "this is where my build steps would run"'
}

Couple things I think to note here:

1. DockerSupport(this) - instantiation passes in the this object, which when running on Jenkins is the CpsScript
2. script.docker.image(imageName).inside {} - the method call isn't directly accessing an available step. It is getting an object (GlobalEnv variable
3. docker is something loaded by a the the CloudBees Docker Pipeline Plugin

@joecarlyon is writing some additional libraries on top of ours, and they are trying to write some unit tests for their libraries.

The problem that seems to be happening now is How do we set up that docker global variable to match what Jenkins does? We are hoping to avoid a huge amount of duplicate copy/paste if we can, but want to know if you have any thoughts on it.

To hopefully give some context around the docker variable - docker is a GlobalVariable that you can see in this Jenkins plugin class. It loads and adds the Docker.groovy (seen here) type in the global docker property.

If you look at the Docker.groovy class, it has a constructor argument that takes a org.jenkinsci.plugins.workflow.cps.CpsScript script. It also has some methods that Docker.groovy class simply wrap other steps loaded by the plugin. It essentially just makes other script calls..

We know we would have to use the existing mocking support in JenkinsPipelineUnit to mock those steps (like withDockerServer, withDockerContainer, etc.) or even mock the docker.inside type call, but would like to use the existing class to mock it's behavior more closely to what is actually happening. What is the best way to handle the creation of that docker property without copying/pasting the entire Docker.groovy class and passing it in as a property?

Also, a sidenote (which is sort related to this issue), in my team's case we want to be able to pass the script object into our global library classes for similar testing for our global libraries. In this case it is passing the script to the constructor of DockerDSL, but we want to be able to pass that test script object around for mocking in our other classes.

[ozangunalp]

Oh thanks for the details, that gives a lot more context. I must say you have a really interesting use case. I am dealing with a similar issue for the ongoing dev in #13 for supporting Declarative Pipelines. Obviously it is not preferable to copy/mock all classes in plugins used such as docker-workflow or declarative pipelines.

I looked a bit on how to mock the docker variable, I think it is possible by some class shadowing, which I accept that is not very clean but pragmatic. I'll try to describe the steps:

• Create a class CspScript in your project that will shadow the original one. Make sure that you got the package name right, it must be org.jenkinsci.plugins.workflow.cps.

• Same for GlobalVariable.

• Set the base class used to instantiate the scripts to CpsScript with helper.scriptBaseClass = CpsScript.class. You must do it before you call setUp on your BasePipelineTest. Avoid using the BasePipelineTestCPS for this example, there is a bug which prevents overriding the baseScriptClass on BasePipelineTestCPS.

• Add the dependency docker-workflow to your project, you need the jar one not the hpi.

• You need to shadow also the classes that Docker accesses. I just found the class DockerRegistryEndpoint. There, you should implement the constructor and the method imageName.

• Prepare a script that uses docker variable etc. and which does not run immediately (returns it self). You will need the script to instantiate docker variable.

• Instantiate the docker variable with new DockerDSL().getValue(script) and set it in the binding as docker variable.

• As you mentioned, you should also register the methods that Docker plugin uses, like withDockerContainer, withDockerRegistry, withDockerServer, withEnv.

• Call the method of your script which executes the pipeline.

The test class should look something like this:

    @Override
    @Before
    void setUp() throws Exception {
        scriptRoots += 'src/test/jenkins'
        helper.scriptBaseClass = CpsScript.class
        super.setUp()
        binding.setVariable('env', [DOCKER_REGISTRY_URL:"https://docker.example.com"])
        helper.registerAllowedMethod("withDockerContainer", [Map, Closure], null)
    }

    @Test
    void should_run_with_docker_dsl() throws Exception {
        def script = loadScript("job/exampleJob.jenkins")
        def docker = new DockerDSL().getValue(script)
        binding.setVariable('docker', docker)
        script.execute()
        printCallStack()
    }

After this experience there are several things that we can contribute to the framework, namely the default script base class can be the shadowed class of CspScript and the helper can let the user to register callbacks to set global vars that require the script itself.

Hope that helps. Let me know if you can make it work.

[joecarlyon]

Sorry for the late reply. I've been pulled off, but was able to work on this yesterday. Copying your code, I was able to get past the Docker problem. Thank you.

[ozangunalp]

Great news @joecarlyon

I am going to leave the issue open because it is not a real fix.

I am working on a solution to ease testing scripts using plugins such as docker. Stay tuned ;)

[joecarlyon]

Looking forward to it!

[roderickrandolph]

Is there a full working example of this workaround on GitHub that I can take a look at? I'm having trouble figuring out how CpsScript and GlobalVariable were shadowed. If someone has a full unit test of how this works and doesn't mind sharing I would be forever grateful to them. Thank you!

[roderickrandolph]

I got it working. In case others run into a similar problem my mistake was trying to shadow under src/test/groovy instead of src/test/java which caused: java.lang.NoClassDefFoundError: org/jenkinsci/plugins/workflow/cps/GlobalVariable

I ended up with 3 shadow classes as @ozangunalp mentioned above. They look like this:

// File: src/test/java/org/jenkinsci/plugins/workflow/cps/CpsScript.java
package org.jenkinsci.plugins.workflow.cps;

import com.cloudbees.groovy.cps.SerializableScript;
import java.io.IOException;

public abstract class CpsScript extends SerializableScript {}

// File: src/test/java/org/jenkinsci/plugins/workflow/cps/GlobalVariable.java
package org.jenkinsci.plugins.workflow.cps;

import hudson.ExtensionPoint;

public abstract class GlobalVariable implements ExtensionPoint {}

// File: src/test/java/org/jenkinsci/plugins/docker/commons/credentials/DockerRegistryEndpoint.java
package org.jenkinsci.plugins.docker.commons.credentials;

import hudson.model.AbstractDescribableImpl;
import java.io.IOException;
import java.net.URL;

public class DockerRegistryEndpoint extends AbstractDescribableImpl<DockerRegistryEndpoint> {

  private final String url;
  private final String credentialsId;

  public DockerRegistryEndpoint(String url, String credentialsId) {
    this.url = url;
    this.credentialsId = credentialsId;
  }

  public URL getEffectiveUrl() throws IOException {
    if (url != null) {
      return new URL(url);
    } else {
      return new URL("https://index.docker.io/v1/");
    }
  }

  public String imageName(String userAndRepo) throws IOException {
    if (userAndRepo == null) {
      throw new IllegalArgumentException("Image name cannot be null.");
    }
    if (url == null) {
      return userAndRepo;
    }
    URL effectiveUrl = getEffectiveUrl();

    StringBuilder s = new StringBuilder(effectiveUrl.getHost());
    if (effectiveUrl.getPort() > 0 && effectiveUrl.getDefaultPort() != effectiveUrl.getPort()) {
      s.append(':').append(effectiveUrl.getPort());
    }
    if (userAndRepo.startsWith(String.valueOf(s))) {
      return userAndRepo;
    }
    return s.append('/').append(userAndRepo).toString();
  }

}

[jdoshi1]

@ozangunalp @joecarlyon @roderickrandolph - I'm not sure if this is the right place to ask this question...but would be great if anyone could help me.

All the above code is working until it fails on - https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/docker/workflow/Docker.groovy#L142 where it tries to run the command with the supplied args but the docker.script.sh() generates null and I get NPE on null.trim()

Here is the code:

String args = "-e BLAH"
def Img = docker.image("someImg:latest")
Img.pull()
Img.withRun(args) { c ->
    sh "docker logs -f ${c.id}"
}

Any idea on how can I solve this?

[baloo42]

@ozangunalp Can you share a complete example?

[g3n35i5]

@ozangunalp are there any updates on this topic? I have pretty much the exact same setup as @mkobit. Is there a better solution to the problem yet or is the shadowing solution still the way to go?