Re-think test organization (Java/JDK matrix vs tasks with toolchains)

[TWiStErRob]

trafficstars

Currently JVM setup and AGP/Gradle setup is done at a Job Matrix level.

Alternatively it could be done via multiple test tasks, lots of configurations. The CI matrix could be kept, but instead of changing the whole environment, it could just pick the right tasks. This could mean that there's no need to edit gradle.properties locally when developing. Just pick the right test task and bam.

This would also help separating Unit tests from integration tests.

Motivation: a test failed 13 times, even though there's one problem: Java 8.

[TWiStErRob]

https://github.com/gradle-nexus/publish-plugin/blob/ab77ce3d5321b7f43bf0fd4e5bc18e184424c29c/build.gradle.kts#L100-L111

[TWiStErRob]

This recurred again with Java 11 v 17

https://github.com/TWiStErRob/net.twisterrob.gradle/pull/502/checks?check_run_id=13867995214

3 tests use specific classes from the AGP/tools jars, which are now (as of 8.2.0-alpha03) compiled to Java 17 class bytecode. The metadata of these can be lowered, so that there's no runtime dependency in Gradle-based tests running older AGPs, but these 3 tests are specifically running Java 17 bytecode on because the matrix otherwise requires 11 for most tests. The root cause is that the project has to be compiled against the latest AGP version to get access to the new APIs and deprecations, and be able to write conditional code. This compileClasspath is exposed to testRuntimeClasspath too, thereby causing Caused by: java.lang.UnsupportedClassVersionError: ... has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0

As a workaround these tests are disabled when running on Java 11:

@DisabledOnJre(JRE.JAVA_11, disabledReason = "https://github.com/TWiStErRob/net.twisterrob.gradle/issues/380")

which is safe for now, because JDK 17 will run them, and they're unit tests independent of matrix-injected AGP/Gradle versions.

[TWiStErRob]

Idea: maintain a single source of truth for supported combinations, including each patch version. Create tasks for each of those, but only run the latest patch / specific combinations on CI via matrix.

Caveat: test if IntelliJ IDEA can handle hundreds of test tasks to pick from when pressing "Play" next to a test.