gradle-stutter

NOTE: As of 0.7.0, gradle-stutter is published to Maven Central

Getting Help or Contributing

IMPORANT: I consider this plugin feature complete and don't spend a lot of time on maintenance due to other time commitments. While, I will eventually get to issues or PRs raised, do not expect a timely response. I'm not trying to be rude or dismissive, I only get back to this project periodically (on the order of months, in many cases). Please set your expectations appropriately as you file issues or open PRs.

Please use the repo's issues for all questions, bug reports, and feature requests.

Why do you care?

When writing a Gradle plugin you often want to run the same suite of tests against all versions you support. The Gradle TestKit gives you the tools to do this, but makes you write the scaffolding for which versions and which tests.

What is it?

gradle-stutter is a Gradle plugin plugin, org.ajoberstar.stutter, which does some common setup for testing Gradle plugins against multiple Gradle versions.

• Extension for specifying Java and Gradle versions that are compatible with your plugin
• Allows specifying different compatible versions for each Java version
• Task to create lock file listing compatible Gradle versions
• Generates a compatibility test task for your suite for each combination of supported Java version and Gradle version
• Uses Gradle's toolchain feature to run each suite under the appropriate JDK

See java-gradle-plugin docs for more details on Gradle's out-of-the-box functionality.

Usage

See the Release Notes for updates on changes and compatibility with Java and Gradle versions.

Applying the Plugin

plugins {
    id 'org.ajoberstar.stutter' version '<version>'
}

Configuration

stutter {
  // if true, only match min/max within that otherwise matches your compatibility specs in each Gradle major version
  sparse = false // defaults to true

  matrices {
    // a matrix of tests running against Java 8
    java8 {
      javaToolchain {
        languageVersion = JavaLanguageVersion.of(8)
      }
      gradleVersions {
        compatibleRange '3.0', '4.0' // include 3.0 <= version < 4.0
        compatibleRange '4.2' // include 4.2 <= version
        compatible '2.14', '1.2' // include 2.14 and 1.12 specifically
        incompatible '3.3' // exclude 3.3 even if included above
      }
    }

    // name can be anything
    jdkNine {
      javaToolchain {
        languageVersion = JavaLanguageVersion.of(9)
      }
      gradleVersions {
        compatibleRange '4.0' // include 4.0 <= version
      }
    }
  }

  // You do have to specify compatible Gradle versions for all Java versions you run Gradle with

  // If you have a lot of tests, or otherwise just don't want to test every Gradle version that you say is compatible,
  // use sparse = true. This will greatly limit the number of versions you test against, but should do the job of
  // verifying compatibility.
  // e.g.  compatible '2.14' and compatibleRange '3.0'
  //       matches '2.14', '3.0', '3.5.1', '4.0', '4.7' (presuming 4.7 is the latest available 4.x)
}

Lock file

Lock files will be generated/used from the <project>/stutter.lockfile directory.

Lock files are generated with the stutterWriteLocks task based on the configuration of the stutter extension (see above).

./gradlew stutterWriteLocks

Tests

The plugin adds a compatTest source set that is configured via java-gradle-plugin as the source set for your plugin tests. This means you can leverage withPluginClasspath on the GradleRunner for your tests. The test kit dependency is not added by this plugin.

The following tasks are available:

• One compatTest<matrix>Gradle<version> (e.g. compatTestJava8Gradle7.3.3) task per supported version (based on the compatible lock file -- see above for details)

• A matrix-level compatTest<matrix> (e.g. compatTestJava8) task that depends on all specific Gradle versions tasks for that matrix. This allows you to run all of the tests for a given JDK matrix via one convenient task.

• An overall compatTest which depends on all of the matrix-level tasks. This allows you to run all tests for all JDKs and Gradle versions.

• check does not depend on compatTest by default, but you can add a dependency easily

  tasks.named("check") {
    dependsOn(tasks.named("compatTest"))
  }
  

Your tests should reference the compat.gradle.version system property when they specify a version on the GradleRunner:

GradleRunner.create()
    .withGradleVersion(System.getProperty("compat.gradle.version"))
    //...

Migrating from 0.6.0

• Gradle must be run with Java 11 or higher
• Lock files have moved from .stutter/ to stutter.lockfile (you should delete the .stutter/ directory)
• sparse is now the default, you can still set it to false if you prefer
• check does not depend on compatTest anymore. This gives you more flexibilty as to which tests are run by default.
• Syntax changes in the stutter extension:

From:

stutter {
  sparse = true

  java(8) {
    compatibleRange '3.0', '4.0'
    compatibleRange '4.2'
    compatible '2.14', '1.2'
    incompatible '3.3'
  }

  java(9) {
    compatibleRange '4.0'
  }
}

To:

stutter {
  sparse = true

  matrices {
    java8 {
      javaToolchain {
        languageVersion = JavaLanguageVersion.of(8)
      }
      gradleVersions {
        compatibleRange '3.0', '4.0'
        compatibleRange '4.2'
        compatible '2.14', '1.2'
        incompatible '3.3'
      }
    }

    java9 {
      javaToolchain {
        languageVersion = JavaLanguageVersion.of(9)
      }
      gradleVersions {
        compatibleRange '4.0'
      }
    }
  }
}

Finding versions of grgit

Newest versions are on Maven Central

As of 0.7.0, gradle-stutter is published to Maven Central and not the Gradle Plugin Portal, but since the portal proxies Maven Central you can still access it through the portal. The only side effect is that the portal will no longer list the latest version. Use this repo or search.maven.org to find the latest version.

Old versions from Bintray/JCenter

This project was previously uploaded to JCenter, which was deprecated in 2021.

In the event that JCenter is unavailable and acess to past versions (0.6.0 and earlier) is needed, I've made a Maven repo available in bintray-backup. Add the following to your repositories to use it.

maven {
  name = 'ajoberstar-backup'
  url = 'https://ajoberstar.org/bintray-backup/'
}

Made possible by lacasseio/bintray-helper in case you have a similar need to pull your old Bintray artifacts.

Acknowledgements

Thanks to all of the contributors.

Alternatives

TestKit (out-of-the-box)

TestKit is built into Gradle, so it should be your first consideration. It provides a good interface to kick off full test builds and verify the output/tasks that ran. This tends to be a far more effective way to test Gradle plugin code than unit testing, due to the complexity of Gradle.

On top of TestKit, Stutter provides a convenient way to run a test suite against multiple versions of Gradle. This is very useful for verifying compatibility.

GradleTest

The org.ysb33r.gradletest plugin is optimized for creating and testing sample projects without you having to directly interact with TestKit. These can also be tested across multiple versions of Gradle providing helpful compatibility verification. GradleTest also provides the ability to test Gradle versions far older than Stutter does.

Stutter leaves the user to decide how to leverage TestKit, it just helps provide the ability to test multiple versions with the same suite. If your use case is more centered around samples or full project tests, GradleTest may be a better fit.