Exception in code generation on Java 22 - NoSuchMethodError: void sun.misc.Unsafe.ensureClassInitialized

[edward3h]

trafficstars

I created a minimal Gradle project to demonstrate the issue. https://github.com/edward3h/pkl-java22-unsafe

When using the pkl code generator in Gradle, on Java 22, I get an exception where this is the root cause:

Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoSuchMethodError: 'void sun.misc.Unsafe.ensureClassInitialized(java.lang.Class)' [in thread "Execution worker Thread 2"]
        at org.pkl.thirdparty.truffle.api.library.LibraryFactory.ensureLibraryInitialized(LibraryFactory.java:384)
        at org.pkl.thirdparty.truffle.api.library.LibraryFactory.getUncached(LibraryFactory.java:364)
        at org.pkl.thirdparty.truffle.api.library.LibraryFactory.<init>(LibraryFactory.java:210)
        at org.pkl.thirdparty.truffle.api.interop.InteropLibraryGen.<init>(InteropLibraryGen.java:178)
        at org.pkl.thirdparty.truffle.api.interop.InteropLibraryGen.<clinit>(InteropLibraryGen.java:169)
        at org.pkl.thirdparty.truffle.api.library.LibraryFactory.loadGeneratedClass(LibraryFactory.java:791)
        at org.pkl.thirdparty.truffle.api.library.LibraryFactory.resolveImpl(LibraryFactory.java:740)
        at org.pkl.thirdparty.truffle.api.library.LibraryFactory.resolve(LibraryFactory.java:733)
        at org.pkl.thirdparty.truffle.api.interop.InteropLibrary.<clinit>(InteropLibrary.java:2941)
        at org.pkl.thirdparty.truffle.polyglot.PolyglotValueDispatch.<clinit>(PolyglotValueDispatch.java:170)
        at org.pkl.thirdparty.truffle.polyglot.PolyglotImpl.initialize(PolyglotImpl.java:166)
        at org.pkl.thirdparty.graalvm.polyglot.impl.AbstractPolyglotImpl.setConstructors(AbstractPolyglotImpl.java:288)
        at org.pkl.thirdparty.graalvm.polyglot.Engine$1.loadAndValidateProviders(Engine.java:1107)
        at org.pkl.thirdparty.graalvm.polyglot.Engine$1.run(Engine.java:1067)
        at org.pkl.thirdparty.graalvm.polyglot.Engine$1.run(Engine.java:1061)
        at org.pkl.thirdparty.graalvm.polyglot.Engine.initEngineImpl(Engine.java:1061)
        at org.pkl.thirdparty.graalvm.polyglot.Engine$ImplHolder.<clinit>(Engine.java:143)
        at org.pkl.thirdparty.graalvm.polyglot.Engine.getImpl(Engine.java:367)
        at org.pkl.thirdparty.graalvm.polyglot.Engine$Builder.build(Engine.java:665)
        at org.pkl.core.runtime.VmUtils.<clinit>(VmUtils.java:74)

My projects work ok on Java 21.

[holzensp]

This is a bug report for GraalVM/Truffle, I'm afraid. Pkl runs on GraalVM 23, which is compatible with JDK 17-21. I expect this to simply go away when we bump to a future GraalVM version that supports Java 22.

[sin-ack]

FYI, based on this page it appears that Truffle 24+ supports Java 23 now: https://docs.oracle.com/en/graalvm/jdk/23/docs/release-notes/#platform-and-distributions

[odenix]

I think the problem is that Truffle 24+ doesn't support older Java versions. I have some ideas on how to support a wider range of Java versions and avoid getting stuck on EOL'd Truffle/Graal versions. I'll start a discussion after the 0.27 release.

[zaninime]

I just bumped into this. I "solved" it by forcing the resolution of the two GraalVM dependencies to 24.1.1 in my project.

What could be blocking a jump from GraalVM 23.0.6 to 24 in general? Is it just a matter of updating the version here and the hashes below?

[odenix]

@zaninime What's blocking the jump is that Pkl supports JDK 17+, whereas GraalVM 24 requires JDK 21+.

[zaninime]

Understood. What's the policy for Pkl on supporting older Java versions? Is it "last two LTS versions"?

[odenix]

I haven't heard an official statement, but Pkl only raised the baseline from 11 to 17 a couple of months ago.

Without any changes, Pkl will always stay years behind an actively maintained GraalVM version, which is undesirable for many reasons. I can think of two solutions:

1. Change Pkl's release model to tip and tail. This is the change that Oracle would like to see in the Java OSS community. A good example for a project that has adopted tip and tail is Spring Boot.
2. Require the latest Java LTS release if the Pkl interpreter itself (not just the Java language binding) is run as a Java library. I think this solution could work very well for Pkl, especially if it offered a native library in addition to the native executable.

[zaninime]

I guess we should move this to a discussion as we're already diverging from the original issue.

I personally find value in embedding the interpreter in apps, to be able to treat the Pkl input as untrusted user input, or put another way, not to rely on an external process to do the validation and produce a well-formed JSON, YAML, etc. for consumption by the software piece that initially needed it. It's as if YAML libraries required anchors and pointers in inputs to be resolved and expanded before consuming them (on another scale).

Maybe the potential native library should be used also in Java apps, then? If so, there shouldn't be a need for a specific JDK/JRE at all for consumers.

[odenix]

I think there should be a Java language binding, eventually superseding pkl-core, that enables Java users to transparently run the Pkl interpreter as a Java library, native library, or external process. If users choose to run the interpreter as a Java library (*), it should hopefully be OK to require the latest LTS JDK version. This would enable the interpreter to

• use the latest LTS JDK features.
• use the latest GraalVM version (which requires the latest LTS JDK version).
• support the latest LTS and non-LTS JDK versions (which requires using the latest GraalVM version and is what this issue is about).

(*) Running the interpreter as just another Java library is extremely slow and heavy on the JVM in terms of class loading and garbage collection. It's also not an option for the CLI and other language bindings, which shouldn't have to face the downsides of an outdated GraalVM version just because this option is offered to Java users.

[bioball]

How would a user on an incompatible Java distribution use the Java library? Does this mean they would need to spawn a sub-process like we do in pkl-go and pkl-swift?

[odenix]

How would a user on an incompatible Java distribution use the Java library? Does this mean they would need to spawn a sub-process like we do in pkl-go and pkl-swift?

Yes, but the Java language binding could be further improved to minimize the impact of this change. It could transparently support consuming Pkl data in the following ways:

1. run interpreter as a Java library (if a compatible Java distribution is used)
2. run interpreter as a native library
3. run interpreter as an external process
4. read Pkl binary data instead of running the interpreter
   • for use cases where Pkl isn't deployed, e.g., service configuration
   • can use 1-3 at development time
   • could support limited runtime configuration For example, code generated by codegen-java could handle @EnvVar { name = "PORT" } port: Int? declared in Pkl schema.

The Java language binding could also make pkl-executor unnecessary.

[bioball]

I'm in favor of having a Java library that uses message passing (just like our other language bindings), but I'm wary of using the latest LTS. I'm worried that the workarounds proposed to users that aren't on the latest LTS would impede adoption from Java users; spawning an external process generally isn't a great story, and calling a native library from Java through JNI has its own problems (including the fact that our native library/executable is slower than Pkl run directly on the JVM)

read Pkl binary data instead of running the interpreter

I definitely want this! Regardless of whether our Java library model changes or not.

[odenix]

including the fact that our native library/executable is slower than Pkl run directly on the JVM

If we're talking about a regular JVM here (not GraalVM), this indicates that something is wrong with the Truffle interpreter and/or native image. It could also be related to networking, memory starvation/garbage collection, etc. In any case, this should be thoroughly investigated. Without some programs to benchmark, it's impossible for the community to help with this investigation.

The first thing I'd try, because it is so easy to do, is to play with memory/GC options and build the native image with --gc=G1 and various other options (latest CPU, highest optimization level, PGO, etc.). FWIW, for the few microbenchmarks I've run, I've noticed that the release executable is about 10% slower than the dev executable (-Ob) on my Windows laptop.

I'm worried that the workarounds proposed to users that aren't on the latest LTS would impede adoption from Java users

Perpetually staying on GraalVM/Truffle versions that are several years old and unsupported isn't a model for success. Running a Truffle interpreter on a regular JVM should be the exception, not the norm, and good engineering can get us there.

calling a native library from Java through JNI has its own problems

A Pkl interpreter that exclusively speaks the binary protocol has a tiny API surface. Additionally, native interop gets much easier with FFM, which was finalized in Java 22 and is already experimentally supported by native-image.

A year from now, supported GraalVM/Truffle versions will require Java 25 LTS. If nothing changes, the Pkl interpreter will be stuck on Java 17 LTS and GraalVM 23 (which is EOL) for years to come. This feels totally wrong.

If the Pkl interpreter (I'm not talking about the Java language binding here) must continue to support Java 17 LTS, the only viable option I see is to adopt a tip & tail release model. But first the premise should be scrutinized.