kotlinx-benchmark icon indicating copy to clipboard operation
kotlinx-benchmark copied to clipboard

Support setting up different toolchains for different benchmark configurations

Open fzhinkin opened this issue 1 year ago • 0 comments
trafficstars

(Spin-off from the #176)

When working on performance improvements, sometimes there's a need to validate performance with different runtime versions. It would be nice to support per-configuration runtime version selection. Something like:

configurations {
    create("jdk17") {
        mode = "avgt"
        outputTimeUnit = "ns"
        include("MyPreciousBenchmark")
        jvmToolchain {
            languageVersion.set(JavaLanguageVersion.of(17))
        }
    }
    create("jdk21") {
        mode = "avgt"
        outputTimeUnit = "ns"
        include("MyPreciousBenchmark")
        jvmToolchain {
            languageVersion.set(JavaLanguageVersion.of(21))
        }
    }
    create("nodeJs21") {
        mode = "avgt"
        outputTimeUnit = "ns"
        include("MyPreciousBenchmark")
        nodeJsVersion = "21.0.0"
    }
}

However, there are a few obvious issues:

  • configurations are not bound to a particular target, so the example above looks weird;
  • it's unclear how to handle scenarios where a user specified a lower JDK version then a project use (i.e. that JDK won't be able to load compiled classes).

Not suggesting a particular solution, filled the issue to ensure the idea won't be lost.

fzhinkin avatar Feb 06 '24 15:02 fzhinkin