oracle
Classloader problem when using Graal JS within a custom Maven plugin
Classloader problem
When building a custom Maven plugin that accesses a polyglot Context, execution within the graalvm of this built plugin in a second Maven project fails:
java.lang.IllegalStateException: No language and polyglot implementation was found on the classpath. Make sure the truffle-api.jar is on the classpath.
at org.graalvm.polyglot.Engine$PolyglotInvalid.noPolyglotImplementationFound (Engine.java:954)
at org.graalvm.polyglot.Engine$PolyglotInvalid.createHostAccess (Engine.java:885)
at org.graalvm.polyglot.Engine$Builder.build (Engine.java:564)
at org.graalvm.polyglot.Context$Builder.build (Context.java:1725)
This can be fixed by setting the current thread's classloader:
Thread.currentThread().setContextClassLoader(Context.class.getClassLoader());
... polyglot code ...
Executing a main method (via Graal's java) compiled in the same library does not require this hack, however. Hence, the issue appears to result from the combination of Graal JS with the Maven classloading mechanism.
Possibly related issues are:
- Graal Issue 2462
-
Graal JS Issue 182: A comment mentions the
ContextBuilder.hostClassLoadermethod as a possible solution to that issue. This is not a solution in the present case.
How to use maven-compiler-plugin
Unfortunately, this issue about runtime behaviour is wrapped up in questions about the proper build-time environment, in particular, when the goal is to:
- use the
maven-compiler-pluginto compile the project accessing the polyglotContext, and - rely on Graal VM for execution without bundling
org.graalvm.js:jsinto the application.
It was difficult to understand which of the examples might apply and dedicated examples for the following simple, standard cases might be helpful to others as well.
1.) Plain, non-Graal compiler
Using the Java 8 or 11 compiler that ships with maven-compiler-plugin succeeds if org.graalvm.js:js is included as a dependency with either the <optional>true</optional> flag or <scope>provided</scope>. The Graal JS library is not bundled into the application and the a simple test can be successfully executed using Graal's java command.
Is that the correct way of compiling?
2.) Graal's javac compiler
When enabling use of Graal's own javac by means of <forceJavacCompilerUse>true</forceJavacCompilerUse>, the org.graalvm.js:js dependency is still required.
This probably means that additional compiler options need to be provided. Which ones?
Your help on the compile process might full well be enough to make my problem evaporate!
Many thanks in advance,
M.
Environment
java --version
openjdk 11.0.12 2021-07-20
OpenJDK Runtime Environment GraalVM CE 21.2.0 (build 11.0.12+6-jvmci-21.2-b08)
OpenJDK 64-Bit Server VM GraalVM CE 21.2.0 (build 11.0.12+6-jvmci-21.2-b08, mixed mode, sharing)
javac --version # javac points at graal vm's bin/javac
javac 11.0.12
mvn --version
Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: /usr/share/maven-3.8.1
Java version: 11.0.12, vendor: GraalVM Community, runtime: /usr/lib/jvm/graalvm-ce-java11-linux-amd64-21.2.0
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-142-generic", arch: "amd64", family: "unix"
Full Example
Custom Plugin
<?xml version="1.0" encoding="UTF-8"?>
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>com.esc.mvn</groupId>
<artifactId>graal-test-maven-plugin</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>maven-plugin</packaging>
<properties>
<graalvm.version>21.2.0</graalvm.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.maven.plugin-tools</groupId>
<artifactId>maven-plugin-annotations</artifactId>
<version>3.6.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-api</artifactId>
<version>3.8.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js</artifactId>
<version>${graalvm.version}</version>
<!-- both work -->
<scope>provided</scope>
<!-- <optional>true</optional> -->
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-plugin-plugin</artifactId>
<version>3.6.1</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<!-- just using Graal's bin/javac is not enough, must probably add more compiler
options -->
<!-- <forceJavacCompilerUse>true</forceJavacCompilerUse> -->
<!-- both work -->
<release>11</release>
<testRelease>11</testRelease>
<!-- <source>1.8</source> -->
<!-- <target>1.8</target> -->
<!-- <testSource>1.8</testSource> -->
<!-- <testTarget>1.8</testTarget> -->
<compilerArgs>
<arg>-parameters</arg>
</compilerArgs>
<testCompilerArgs>
<arg>-parameters</arg>
</testCompilerArgs>
</configuration>
</plugin>
<plugin>
<artifactId>maven-plugin-plugin</artifactId>
<executions>
<execution>
<id>default-descriptor</id>
<phase>process-classes</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
package com.esc.graaltest;
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugins.annotations.LifecyclePhase;
import org.apache.maven.plugins.annotations.Mojo;
@Mojo(name = "compile",
requiresProject = false,
defaultPhase = LifecyclePhase.COMPILE,
threadSafe = false)
public class TestMojo extends AbstractMojo {
@Override
public void execute() {
new Test().test();
}
}
package com.esc.graaltest;
import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Value;
public class Test {
static public void main(String[] args) {
new Test().test();
}
public void test() {
ClassLoader cl = null;
if("true".equals(System.getProperty("HACK"))) {
// code below fails without this HACK
cl = Thread.currentThread().getContextClassLoader();
Thread.currentThread().setContextClassLoader(Context.class.getClassLoader());
}
Context context = Context.newBuilder("js").build();
Value o = context.eval("js", "'hi there'");
System.out.println("Graal JS says: " + o.asString());
if("true".equals(System.getProperty("HACK")))
Thread.currentThread().setContextClassLoader(cl);
}
}
Running the main Test
No problem here. Starting in the directory containing the POM:
cd target/classes
java com.esc.graaltest.Test
Graal JS says: hi there
Using the Custom Plugin
In a second directory, we have a POM using the custom plugin built before:
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>com.esc.test</groupId>
<artifactId>graal-test</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<build>
<plugins>
<plugin>
<groupId>com.esc.mvn</groupId>
<artifactId>graal-test-maven-plugin</artifactId>
<version>1.0.0-SNAPSHOT</version>
<executions>
<execution>
<id>test</id>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
With the classloader hack, the project builds correctly:
mvn -D HACK=true clean compile
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------< com.esc.test:graal-test >-----------------------
[INFO] Building graal-test 1.0.0-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- graal-test-maven-plugin:1.0.0-SNAPSHOT:compile (test) @ graal-test ---
Graal JS says: hi there
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.699 s
[INFO] Finished at: 2021-08-12T13:10:34-04:00
[INFO] ------------------------------------------------------------------------
Without the classloader hack, the project fails:
mvn -e -D HACK=false clean compile
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------< com.esc.test:graal-test >-----------------------
[INFO] Building graal-test 1.0.0-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- graal-test-maven-plugin:1.0.0-SNAPSHOT:compile (test) @ graal-test ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.132 s
[INFO] Finished at: 2021-08-12T13:19:11-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.esc.mvn:graal-test-maven-plugin:1.0.0-SNAPSHOT:compile (test) on project graal-test: Execution test of goal com.esc.mvn:graal-test-maven-plugin:1.0.0-SNAPSHOT:compile failed: No language and polyglot implementation was found on the classpath. Make sure the truffle-api.jar is on the classpath. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.esc.mvn:graal-test-maven-plugin:1.0.0-SNAPSHOT:compile (test) on project graal-test: Execution test of goal com.esc.mvn:graal-test-maven-plugin:1.0.0-SNAPSHOT:compile failed: No language and polyglot implementation was found on the classpath. Make sure the truffle-api.jar is on the classpath.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:566)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution test of goal com.esc.mvn:graal-test-maven-plugin:1.0.0-SNAPSHOT:compile failed: No language and polyglot implementation was found on the classpath. Make sure the truffle-api.jar is on the classpath.
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:566)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.IllegalStateException: No language and polyglot implementation was found on the classpath. Make sure the truffle-api.jar is on the classpath.
at org.graalvm.polyglot.Engine$PolyglotInvalid.noPolyglotImplementationFound (Engine.java:954)
at org.graalvm.polyglot.Engine$PolyglotInvalid.createHostAccess (Engine.java:885)
at org.graalvm.polyglot.Engine$Builder.build (Engine.java:564)
at org.graalvm.polyglot.Context$Builder.build (Context.java:1725)
at com.esc.graaltest.Test.test (Test.java:24)
at com.esc.graaltest.TestMojo.execute (TestMojo.java:16)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:566)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
Thank you for reporting this issue @MatthiasHild. We will take a look into it and get back to you.
Hi @MatthiasHild. Please replace
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js</artifactId>
<version>${graalvm.version}</version>
<!-- both work -->
<scope>provided</scope>
<!-- <optional>true</optional> -->
</dependency>
with
<dependency>
<groupId>org.graalvm.sdk</groupId>
<artifactId>graal-sdk</artifactId>
<version>${graalvm.version}</version>
</dependency>
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js</artifactId>
<version>${graalvm.version}</version>
<scope>runtime</scope>
</dependency>
and you would no longer need to invoke the classloader.
Rodrigo,
Thank you for investigating.
I don't think your proposal is a solution that takes advantage of Graal's full performance potential.
Perhaps, my statement of one of the two goals could have been clearer as I was trying to find a general formulation:
rely on Graal VM for execution without bundling org.graalvm.js:js into the application.
In this specific context, what we want is not to put org.graalvm.js:js onto the classpath (scope runtime does exactly that). We don't want this because it is bad for performance: We want to run within GraalVM and not lose its "runtime compilation" (see logs below).
With your proposal, we get GraalVM's vehement protestations:
[INFO] --- graal-test-maven-plugin:1.0.0-SNAPSHOT:compile (test) @ graal-test ---
[To redirect Truffle log output to a file use one of the following options:
* '--log.file=<path>' if the option is passed using a guest language launcher.
* '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
* Configure logging using the polyglot embedding API.]
[engine] WARNING: The polyglot context is using an implementation that does not support runtime compilation.
The guest application code will therefore be executed in interpreted mode only.
Execution only in interpreted mode will strongly impact the guest application performance.
For more information on using GraalVM see https://www.graalvm.org/java/quickstart/.
To disable this warning the '--engine.WarnInterpreterOnly=false' option or use the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property.
Graal JS says: hi there
Finally, a comment on a detail of your proposal, namely the splitting the js and graal-sdk dependencies. I mention this side issue only to make sure there are no misunderstandings, but it is not central to the problem. Your splitting of the dependencies makes not effective difference in the present scenario. Yes, org.graalvm.sdk:graal-sdk is all we need to compile Test ( and TestMojo in the presence of the hack logic) but you don't even need it with scope compile - provided is enough. The problem is the runtime dependence on org.graalvm.js:js. Once you include it at runtime, you might as well used it at compile time and then ditch the explicit mention of the transitive org.graalvm.sdk:graal-sdk dependency. That's why I used the stronger dependency on org.graalvm.js:js to make experimentation easier - I had explored your proposal before. Of course, I made this depencency provided to avoid the performance degradation.
Best wishes,
M.
BTW, the comment on Graal JS Issue 182 concerning the new ContextBuilder.hostClassLoader method may intend to suggest a follow-on hack around the Thread.currentThread().setContextClassLoader(Context.class.getClassLoader()) hack already mentioned by the reporter of that issue. When changing the current thread's class loader (to satisfy org.graalvm.polyglot.Engine), instantiation of Java objects from within Javascript now uses the wrong class loader - which may be offset by setting ContextBuilder.hostClassLoader to the current thread's original class loader that was so rudely replace by the first hack. Oh my!
In the upcoming GraalVM 22.3, the language classes should now always be found and successfully loaded regardless of the current context class loader, as long as the org.graalvm.polyglot.* classes are loaded by the correct class loader.
mvn -e -D HACK=false clean compile worked for me using a recent developer build.
I hope this fixes the issue for you. Feel free to reopen if you still have trouble with 22.3.x.
