runPluginVerifier returns returns with exit code 0 even if it finds some errors

[tpasternak]

What happened?

runPluginVerifier exited with 0 despite spotting some errors

Relevant log output or stack trace

No response

Steps to reproduce

1. Create a hello world plugin
2. Do something that contains error spot by plugin verifier (for example mark it dynamic, but implement a non-dynamic extension point)
3. Execute ./gradlew :runPluginVerifier

Result: The error message is print correctly, but the exit code is 0, so it's impossible to use runPluginVerifier to implement a CI guard for some features.

Gradle IntelliJ Plugin version

1.16.0

Gradle version

unrelated

Operating System

None

Link to build, i.e. failing GitHub Action job

No response

[hsz]

As discussed, the problem is categorized as Internal API usages. Therefore, NOT_DYNAMIC doesn't make the runPluginverifier task to fail.

An example issue report looks like:

Plugin org.jetbrains.plugins.template:1.11.2 against IC-232.10227.8: Compatible. 1 usage of internal API
Internal API usages (1): 
    #Internal interface com.intellij.ide.ApplicationInitializedListener reference
        Internal interface com.intellij.ide.ApplicationInitializedListener is referenced in org.jetbrains.plugins.template.listeners.MyApplicationInitializedListener. This interface is marked with @org.jetbrains.annotations.ApiStatus.Internal annotation or @com.intellij.openapi.util.IntellijInternalApi annotation and indicates that the class is not supposed to be used in client code.
    Plugin probably cannot be enabled or disabled without IDE restart: Plugin probably cannot be enabled or disabled without IDE restart because it declares non-dynamic extensions: `com.intellij.applicationInitializedListener`

cc @novotnyr

[novotnyr]

The Internal API Usages and the dynamic eligibility are two different concepts.

Additionally, this might be connected with Plugin Verifier and possibly misleading message output in the console. This has been already fixed.