Allow JBR runtime package of distributions to be used--instead of automatically downloading a random binary VM

[daym]

I'm using IntelliJ IDEA in the GNU Guix Linux distribution.

Guix has a somewhat unusual filesystem setup. Among other things you won't be able to run a random foreign ELF executable because the loader is god knows where (specific example /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/ld-linux-x86-64.so.2). That's because it's taking modularity seriously and so there is no "the" ld.so, there are many (on my dev system right now there are 32 different ones at the same time). There's no /lib in the root directory at all.

In order to make IntelliJ IDEA work nicely, we packaged jbr17, building it from https://github.com/JetBrains/JetBrainsRuntime/ like any other JDK is built. That made IntelliJ IDEA work nicely.

Eventually I got into IntelliJ plugin development. That is where the problems started.

The IntelliJ plugin, understandably, tries to use the JBR runtime in RunIdeBase, and by inheritance also in BuildSearchableOptionsTask, RunIdeTask, RunIdeForUiTestTask, RunIdePerformanceTestTask, RunIdePerformanceTestTask. It makes sense to use the same JVM that is used for running the IDE also when testing the IDE, so that goal is fine.

But none of them work because gradle-intellij-plugin downloads a binary JBR and then runs that, which won't work because it won't find ld.so.

I propose (and have successfully tested) to make gradle-intellij-plugin also heed a JBR VM specified via gradle.properties like this:

org.gradle.java.installations.auto-detect=false
org.gradle.java.installations.fromEnv=IDEA_JDK,JBR17,JBR11

To be clear, why this is so difficult:

1. When you run IntelliJ, Guix's JBR is in PATH and so that gets used to start it. So far so good.
2. When you then open an intellij plugin project (or anything, really), Gradle chooses a suitable Java VM for running Gradle (usually that's not JBR), and then builds whatever. As long as that Java VM is in the Guix profile, that works just fine--and IntelliJ IDEA can be used just fine for regular development.
3. But if you write an IntelliJ plugin yourself, eventually, gradle-intellij-plugin chooses yet another Java VM to use for the gradle tasks buildSearchableOptions, runIde etc. It's NOT using the exact same one as in 1.

It would be nice to enable 3. to use the exact same executables as 1. did.

[JojOatXGME]

The same problem exists on NixOS. There may also be other reasons to change the JVM. For example, I noticed #1452 while looking through the issues. Since 1.13.3 or #473, it does also affect all Test tasks. I am currently looking into updating the workaround for the nix-idea plugin. The old workaround is here:

https://github.com/NixOS/nix-idea/blob/df0a3237a8bd7bc13a31ce2d3d0dbb25d62a8757/build.gradle.kts#L112-L123

The following seems to work as an updated workaround which also changes the JVM of the tests: UPDATE: RunPluginVerifierTask is still not working.

// Override JVM of gradle-intellij-plugin with JVM at `jbr/bin/java`
tasks.withType<RunIdeBase> {
    projectExecutable = jbrExecutable.toString()
}
pluginManager.withPlugin("org.jetbrains.intellij") {
    // Uses `withPlugin` because the following code must run after the gradle-intellij-plugin got applied.
    // We must also use `afterEvaluate`, as the gradle-intellij-plugin uses `afterEvaluate` as well.
    // Otherwise, gradle-intellij-plugin would just overwrite our configuration.
    afterEvaluate {
        tasks.withType<Test> {
            executable = jbrExecutable.toString()
        }
    }
}

Note that the afterEvaluate is a bit strange. Would be nice if gradle-intellij-plugin had better support for changing the path to the runtime. I also thought about using dependency substitution, but I couldn't think of a good solution in this direction.

[JojOatXGME]

I just noticed that RunPluginVerifierTask is still not working with my workaround from above. The problem is that RunPluginVerifierTask.validateRuntimeDir probes the JBR by calling java -version.

https://github.com/JetBrains/gradle-intellij-plugin/blob/cc828c20fb7b4aba9cb6c190148581efabb10bda/src/main/kotlin/org/jetbrains/intellij/tasks/RunPluginVerifierTask.kt#L458-L462

If I use the JBR from NixOS …

tasks.withType<RunPluginVerifierTask> {
    resolvedRuntimeDir = jbrHome
}

I run into the following exception:

Starting the IntelliJ Plugin Verifier 1.364
Verification reports directory: /home/nixos/nix-idea/build/reports/pluginVerifier
2024-03-11T12:29:36 [main] INFO  verification - Reading IDE /home/nixos/.cache/pluginVerifier/ides/IC-233.14475.9
2024-03-11T12:29:36 [main] INFO  c.j.p.options.OptionsParser - Reading IDE from /home/nixos/.cache/pluginVerifier/ides/IC-233.14475.9
2024-03-11T12:29:36 [main] INFO  c.j.p.options.OptionsParser - Using Java runtime from /home/nixos/nix-idea/jbr
Exception in thread "main" java.lang.IllegalArgumentException: JDK at /home/nixos/nix-idea/jbr does not have any indication of the JDK build number. Please create a file <JDK home>/version.txt that contains a string of JDK version such as 1.8.0 or 11
	at com.jetbrains.pluginverifier.jdk.JdkDescriptorCreator.createJdkDescriptor(JdkDescriptorCreator.kt:41)
	at com.jetbrains.pluginverifier.jdk.JdkDescriptorCreator.createJdkDescriptor(JdkDescriptorCreator.kt:31)
	at com.jetbrains.pluginverifier.jdk.JdkDescriptorCreator.createJdkDescriptor$default(JdkDescriptorCreator.kt:28)
	at com.jetbrains.pluginverifier.jdk.DefaultJdkDescriptorProvider.fromExplicitPath(JdkDescriptorProviders.kt:38)
	at com.jetbrains.pluginverifier.jdk.DefaultJdkDescriptorProvider.getJdkDescriptor(JdkDescriptorProviders.kt:26)
	at com.jetbrains.pluginverifier.ide.IdeDescriptor$Companion.create(IdeDescriptor.kt:63)
	at com.jetbrains.pluginverifier.options.OptionsParser.createIdeDescriptor(OptionsParser.kt:114)
	at com.jetbrains.pluginverifier.options.OptionsParser.createIdeDescriptor(OptionsParser.kt:105)
	at com.jetbrains.pluginverifier.tasks.checkPlugin.DefaultIdeDescriptorParser.parseIdeDescriptors(CheckPluginParamsBuilder.kt:126)
	at com.jetbrains.pluginverifier.tasks.checkPlugin.CheckPluginParamsBuilder.build(CheckPluginParamsBuilder.kt:41)
	at com.jetbrains.pluginverifier.tasks.checkPlugin.CheckPluginParamsBuilder.build(CheckPluginParamsBuilder.kt:27)
	at com.jetbrains.pluginverifier.PluginVerifierMain$main$3$1.invoke(PluginVerifierMain.kt:130)
	at com.jetbrains.pluginverifier.PluginVerifierMain$main$3$1.invoke(PluginVerifierMain.kt:123)
	at com.jetbrains.pluginverifier.tasks.profiling.PluginVerificationProfilingsKt.measurePluginVerification(PluginVerificationProfilings.kt:11)
	at com.jetbrains.pluginverifier.PluginVerifierMain.main(PluginVerifierMain.kt:123)

I am not sure for what the IntelliJ Plugin Verifier needs the JBR, as itself is running on another JVM. Assuming the plugin verifier does not execute the given JBR, but only uses it to check whether classes and methods do exist, maybe we could just avoid the initial probe by the RunPluginVerifierTask?

[hsz]

In the 2.0 release, I've redesigned how the JBR is resolved and how you can configure it for your project. Could we focus on that release? It'd be great if you could migrate your setup to 2.0 and we'll further investigate if anything still need my attention here. How does that sound?

[JojOatXGME]

@hsz I actually tried 2.0 already and my solution got a bit simpler.

https://github.com/NixOS/nix-idea/commit/c912d4fa20aaa440c5b3adebde08d167a0d71052#diff-13ca58d2e28bbe0bf7c4514d75a7ad23898fcf4ae547b4ecbb61ab5e90532fc0 (See gradle/plugins/src/main/kotlin/local.jbr-guidance.gradle.kts)

If I remember correctly also RunPluginVerifierTask was working with 2.0 the last time I tested it.

[hsz]

The runPluginVerifier was renamed to verifyPlugin, but you noticed that already.

Is this solution involving RuntimeAware-based tasks reconfiguration suitable for you? Actually, this is one of reasons why such *Aware strategy was introduced.

BTW, you can also try:

tasks
  .matching { it is RuntimeAware }
  .configureEach { runtimeDirectory = jbrHome }

[hsz]

Ok, I went an extra mile here. ;) Since 2.0.0-beta9, it'll be possible to specify the local path to JetBrains Runtime via dependencies, so in your case that'd be:

dependencies {
    intellijPlatform {
        create(platformType, platformVersion, useInstaller = false)
        jetbrainsRuntimeLocal(jbrHome)
        ...
    }
}

The path provided to jetbrainsRuntimeLocal helper will be passed to the jetbrainsRuntimeLocalInstance configuration and eventually resolved by the JavaRuntimePathResolver which sets it to runtimeDirectory property.

Thanks to that, also runtimeMetadata and runtimeArchitecture are correctly set.

The useInstaller = false is also something introduced in 2.0.0-beta9. By default, installers are resolved from download.jetbrains.com, which are OS- and arch-specific. This means, in their product-info.json, there's a setup (system properties, etc) targeting only specific OS/arch. The non-installer's product-info.json contains a full set of options, which may help you. But I'd start with installer (so rely on a default useInstaller = true flag).

Related commit: https://github.com/JetBrains/intellij-platform-gradle-plugin/commit/3e4277d5a09be9be5fb1e5aa9b38b37b02d8ff7f