intellij-platform-gradle-plugin icon indicating copy to clipboard operation
intellij-platform-gradle-plugin copied to clipboard

Each individual test re-downloads large IDE files from remote repositories - really heavy internet usage!

Open aSemy opened this issue 3 years ago • 2 comments

Describe the bug

When I run VerifyPluginConfigurationTaskSpec locally, the tests are very slow - each taking about 1 minute each. On investigation I found that every test is re-downloading the IDE files.

This downloading takes place in Gradle's configuration phase, which is against best practice https://docs.gradle.org/current/userguide/authoring_maintainable_build_scripts.html#sec:minimize_logic_executed_configuration_phase

Be aware that any code that is not part of a task action will be executed with every single run of the build

To Reproduce

This is most evident in the GitHub Action tests - they should not be taking so long.

Expected behavior

  • Gradle dependency-resolution is used to ensure downloaded files are cached and re-used
  • Files are not downloaded in Gradle's configuration phase

Environment:

Additional context

This is related to

  • #1114
  • #1113
  • #776

aSemy avatar Oct 01 '22 11:10 aSemy

I think this is caused by RunPluginVerifierTask downloading the IDE files into a input property

https://github.com/JetBrains/gradle-intellij-plugin/blob/7f0c94e042a8a8408a85f5708a1956ac53349a76/src/main/kotlin/org/jetbrains/intellij/tasks/RunPluginVerifierTask.kt#L138-L140

I think it should be resolved by extracting the 'download IDE files' functionality out of RunPluginVerifierTask into a separate task. This new task should have the downloaded files in an @OutputFiles or @OutputDirectory property.

The same is probably true of most of the 'download files' functionality that gradle-intellij-plugin has.

aSemy avatar Oct 01 '22 15:10 aSemy

I created a prototype solution #1144, where the download task and plugin-verifier task are separated. I did a very quick test, and this seemed to work for one test. There are some caveats, see the PR for more details.

While working on the prototype I became a bit concerned, because there's no verification of the downloaded files, or whether they have been modified since. So there's a risk of corrupted downloads, and since the files are executed, does this present a security risk? Could the files be modified by another process, which would then be executed at the user's permission level?

Does the API provide checksums?

aSemy avatar Oct 03 '22 16:10 aSemy

Implemented in 2.0. All IDEs used by the IntelliJ Plugin Verifier are now handled through the Gradle dependencies management. It is possible to apply the org.jetbrains.intellij.platform.settings plugin in settings.gradle.kts, which also provides a set for repository helpers to let you add all necessary repositories to the project.

See: https://plugins.jetbrains.com/docs/intellij/tools-intellij-platform-gradle-plugin.html#configuration.dependencyResolutionManagement

hsz avatar Mar 14 '24 15:03 hsz