run tests across multiple IJ versions

[rsmckinney]

My plugin must work with many different versions of IJ; configured via sinceBuild. Yet the tests only run on the version of IJ my plugin compiles against. I'd like to configure my plugin to run tests against a configured set of IJ versions, or by default run against the version corresponding with sinceBuild target in addition to the version compiled against.

On the same subject I'd like to configure the version of IJ to run against. I see there is already some functionality for that via alternativeIdePath, however I'd like to use a version number like 2016.3 instead of an explicit path.

[jansorg]

@rsmckinney This is rather simple with the plugin. Checkout https://github.com/BashSupport/BashSupport/blob/master/build-all-branches.bash as an example. To run with a specific version see https://github.com/BashSupport/BashSupport/blob/master/run-idea-2018.2.bash

[amynbe]

@jansorg this doesn't seem to be (or no longer) working? A search for "ideaVersion" or "ideaBranch" across the repo doesn't return any result either...

[amynbe]

Sorry, it works fine -- the properties have to be defined in gradle.properties. It then works even when calling a task from within IntelliJ and passing the relevant -P arguments

[rsmckinney]

@jansorg thanks for your answer. I don't think my original question was clear. What I'd like is to run tests against versions other than the one my plugin compiles against, however I do not want to build against those other versions as my plugin is optimized to build against a specific newer version and works with older versions using reflection.

My description of the problem is relevant for a plugin distribution targeting more than just one IJ version, which I suspect describes many other plugins.

[hsz]

Running tests using different IntelliJ Platform is now possible with the 2.0 release.

import org.jetbrains.intellij.platform.gradle.IntelliJPlatformType
import org.jetbrains.intellij.platform.gradle.tasks.TestIdeTask

tasks {
  val testPhpStorm by registering(TestIdeTask::class) {
    type = IntelliJPlatformType.PhpStorm
    version = "2023.2.2"
  }

  val testLocalIde by registering(TestIdeTask::class) {
    localPath = file("/Users/hsz/Applications/Android Studio.app")
  }
}

See: https://plugins.jetbrains.com/docs/intellij/tools-intellij-platform-gradle-plugin-tasks.html#testIde