Cannot run `runPluginVerifier` in parallel

[jhonnen]

Describe the bug I'm having troubles parallelizing the build of a repository containing multiple IntelliJ plugins because of runPluginVerifier.

I'm seeing two distinct issues preventing the task to be run in parallel:

1. I want all tasks to share the same downloadDir, so that an IDE is downloaded just once and not for each subproject. But then all tasks download/extract into the same directory concurrently, corrupting it:

   Exception in thread "main" com.jetbrains.plugin.structure.ide.InvalidIdeException: IDE by path '<rootProjectDir>\build\pluginVerifier\IC-2022.2.4' is invalid: IDE directory content is invalid
           at com.jetbrains.plugin.structure.ide.IdeManagerImpl.createIde(IdeManagerImpl.kt:34)
           at com.jetbrains.plugin.structure.ide.IdeManagerImpl.createIde(IdeManagerImpl.kt:25)
           at com.jetbrains.pluginverifier.ide.IdeDescriptor$Companion.create(IdeDescriptor.kt:59)
           at com.jetbrains.pluginverifier.options.OptionsParser.createIdeDescriptor(OptionsParser.kt:78)
           at com.jetbrains.pluginverifier.options.OptionsParser.createIdeDescriptor(OptionsParser.kt:73)
           at com.jetbrains.pluginverifier.tasks.checkPlugin.CheckPluginParamsBuilder.build(CheckPluginParamsBuilder.kt:38)
           at com.jetbrains.pluginverifier.tasks.checkPlugin.CheckPluginParamsBuilder.build(CheckPluginParamsBuilder.kt:23)
           at com.jetbrains.pluginverifier.PluginVerifierMain.main(PluginVerifierMain.kt:125)
   
   

2. Once all IDEs are downloaded, the tasks still cannot be run in parallel because plugin.verifier.home.dir system property is not set by the task. Therefore all tasks modify the same directory in the user.home:

   Exception in thread "main" java.nio.file.FileAlreadyExistsException: <user.home>\.pluginVerifier\extracted-plugins
           at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:87)
           at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
           at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
           at java.base/sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:521)
           at java.base/java.nio.file.Files.createDirectory(Files.java:700)
           at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
           at java.base/java.nio.file.Files.createDirectories(Files.java:753)
           at com.jetbrains.plugin.structure.base.utils.FileUtilKt.createDir(FileUtil.kt:62)
           at com.jetbrains.pluginverifier.PluginVerifierMain.getPluginsExtractDirectory(PluginVerifierMain.kt:80)
           at com.jetbrains.pluginverifier.PluginVerifierMain.main(PluginVerifierMain.kt:116)
   

To Reproduce

• Gradle project with multiple intellij-plugin subprojects
• configure downloadDir/downloadPath to the same location:

  def commonDownloadDir = rootProject.layout.buildDirectory.dir("pluginVerifier")
  subprojects {
      tasks.withType(RunPluginVerifierTask).configureEach {
          downloadDir = commonDownloadDir.map { it.asFile.path }
          downloadPath = commonDownloadDir.map { it.asFile.toPath() }
      }
  }
  

• run task in parallel: gradlew runPluginVerifier --parallel

Expected behavior Parallel execution works without errors, IDE has to be downloaded just once (maybe by extracting the download into a separate task?).

Environment:

• OS: Windows 11
• Gradle IntelliJ Plugin Version 1.13.1
• Gradle Version 8.0.2

[hsz]

Extracting IDEs downloading into a separated task is the way to go.

[hsz]

With the 2.0 release, IDEs used by the IntelliJ Plugin Verifier are managed by the Gradle dependencies mechanism. That will let Gradle handle this scenario with ease.