Scala-cli cannot build native-image if using Java 21 (GraalVM) as default JDK

[carlosedp]

trafficstars

Version(s)

❯ scli --version
Scala CLI version: 1.0.6
Scala version (default): 3.3.1

❯ java -version
openjdk version "21.0.1" 2023-10-17
OpenJDK Runtime Environment GraalVM CE 21.0.1+12.1 (build 21.0.1+12-jvmci-23.1-b19)
OpenJDK 64-Bit Server VM GraalVM CE 21.0.1+12.1 (build 21.0.1+12-jvmci-23.1-b19, mixed mode, sharing)

Describe the bug I use the latest GraalVM (Java 21) as default in my dev workstation. When trying to build a native-image of an app, scala-cli complains the classes were compiled with a newer version of the Java Runtime.

Maybe, scala-cli used my install of GraalVM-Java21 to compile the classes but used it's own GraalVM and native-image tool to build the native image. As seen above, I use the latest GraalVM 21 (with newer naming) and scala-cli installed the native-image tool for GraalVM 22.3.1.

❯ scli package --native-image ./unary_client_scala.scala
Compiling project (Scala 3.3.1, JVM (21))
Compiled project (Scala 3.3.1, JVM (21))

Downloading: Component catalog from www.graalvm.org
Processing Component: Native Image
Downloading: Component native-image: Native Image from github.com
Installing new component: Native Image (org.graalvm.native-image, version 22.3.1)
========================================================================================================================
GraalVM Native Image: Generating 'UnaryClient' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                    (0.0s @ 0.28GB)
Fatal error: java.lang.UnsupportedClassVersionError: UnaryClient has been compiled by a more recent version of the Java Runtime (class file version 65.0), this version of the Java Runtime only recognizes class file versions up to 61.0
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	at java.base/jdk.internal.loader.Loader.loadClass(Loader.java:564)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:467)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageClassLoader.forName(ImageClassLoader.java:296)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageClassLoader.forName(ImageClassLoader.java:292)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageClassLoader.forName(ImageClassLoader.java:301)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:339)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:580)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)
Error: Image build request failed with exit status 1
Error: scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
For more details, please see '/Users/cdepaula/repos/scala/scala-playground/protobuf/unary/.scala-build/stacktraces/1700058696-5117202761185469735.log'

Expected behaviour Maybe have scala-cli use the default installed JDK?

[Gedochao]

Yeah... so the example given is failing because Java 21 (which in your case is graalvm-java21:21.0.1) is used to compile your code, while Java 17 (graalvm-java17:22.3.1) is used to build the GraalVM native image. As you can't build an image with Java 17 when the code has been compiled with Java 21, it's bound to fail.

Why Java 17 is used to build the native image, you might ask? Well, that's because it is the default, and you have not explicitly changed it.

scala-cli --power package --full-help 2>&1 |grep graalvm             
#   -j, --jvm jvm-name                             Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system`
#   --graalvm-java-version java-major-version  GraalVM Java major version to use to build GraalVM native images (17 by default)
#   --graalvm-version version                  GraalVM version to use to build GraalVM native images (22.3.1 by default)
#   --graalvm-jvm-id jvm-id                    JVM id of GraalVM distribution to build GraalVM native images (like "graalvm-java17:22.0.0")
#   --graalvm-args string*                     Pass args to GraalVM

Which I definitely see isn't great UX right there. So first things first, to make your build work, you can just do this:

scala-cli package --power --native-image --graalvm-java-version 21 --graalvm-version 21.0.1 unary_client_scala.scala
# (...)

This way the --jvm being used (which defaults to your JAVA_HOME, so 21) matches the GraalVM used to build the Java version. --graalvm-version has to be passed as well, as there's no community build GraalVM 22.3.1 for Java 21. Which makes it even more tricky, as particular GraalVM versions have varying subsets of supported Java and architectures (the lack of M1 releases for GraalVM >22.3.1 is precisely why we're currently locked to that version by default).

Now, I completely agree we could try to make this smarter:

• if --jvm is explicitly being passed, but graalvm-java-version/--graalvm-jvm-id isn't, we could try to default Graal to match the Java
• we could try to suggest ways to fix this after failing to build a native image, as the current error message isn't very helpful (the --graalvm-* options aren't easy to discover, unless someone greps through the full help with them in mind)

Now, even though in the original example GraalVM is available in JAVA_HOME and could theoretically be used automatically for all of it, I'm not sure if it's reasonable to look for Graal there, verify if it's a GraalVM distribution and then use it. Most users don't use GraalVM as their default Java. But we could check how hard it'd be to actually check for it.

So TL;DR it actually isn't really a bug, as it works as intended, but how it works is pretty far from self-explanatory, let's try to improve the UX of this in the scope of this ticket.

[carlosedp]

Thank you so much for the explanation Piotr! I've opened the issue exactly with this intent of getting a feedback and improve the UX.

Could also use --graalvm-jvm-id graalvm-java21:21.0.1 instead of two params right. I tried passing it thru directives //> using jvm graalvm-java21:21.0.1, but it didn't work.

[Gedochao]

Could also use --graalvm-jvm-id graalvm-java21:21.0.1 instead of two params right.

Yeah.

I tried passing it thru directives //> using jvm graalvm-java21:21.0.1, but it didn't work.

Yeah, that only sets the Java used for compilation to graalvm-java21:21.0.1 (which is what you had already, as it defaults to your JAVA_HOME). The //> using jvm ... directive is an equivalent of the --jvm command line option. It seems we don't have a directive equivalent for --graalvm-java-version/--graalvm-jvm-id at this time.