InternalResourceCache is hard-coded to 'user.home'. Overriding the path fails in tests.

[d-schmidt]

Describe the issue The Graal in our Spring Boot wont deploy or wont test because it requires access to user.home and the override of polyglot.engine.resourcePath doesn't work.

We might override it: JavaDoc But this fails in tests because line https://github.com/oracle/graal/blob/75074233f34a2b73d721157fb6bccd1cc1ac9ce0/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/InternalResourceCache.java#L218 demands versioning. Overriding creates an unversioned path: https://github.com/oracle/graal/blob/75074233f34a2b73d721157fb6bccd1cc1ac9ce0/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/InternalResourceRoots.java#L215

Without override it is hardcoded to user.home: https://github.com/oracle/graal/blob/75074233f34a2b73d721157fb6bccd1cc1ac9ce0/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/InternalResourceRoots.java#L267

Steps to reproduce the issue

• embedd GraalVM JS
• run it in a container without access to user.home
• try to override the polyglot.engine.resourcePath System.setProperty("polyglot.engine.resourcePath", System.getProperty("java.io.tmpdir"));

Describe GraalVM and your environment: OpenJDK 17 and Spring Boot 3.1.7 and:

<dependency>
    <groupId>org.graalvm.js</groupId>
    <artifactId>js-scriptengine</artifactId>
    <version>23.1.2</version>
</dependency>
<dependency>
    <groupId>org.graalvm.polyglot</groupId>
    <artifactId>js-community</artifactId>
    <version>${graalvm.version}</version>
    <type>pom</type>
    <scope>23.1.2</scope>
</dependency>

More details Add any other information about the problem here. Especially important are stack traces or log output. Feel free to link to gists or to screenshots if necessary.

Default home fail:

Caused by: java.nio.file.AccessDeniedException: /app/.cache
	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(Unknown Source)
	at java.base/java.nio.file.Files.createDirectory(Unknown Source)
	at java.base/java.nio.file.Files.createAndCheckIsDirectory(Unknown Source)
	at java.base/java.nio.file.Files.createDirectories(Unknown Source)
	at com.oracle.truffle.polyglot.InternalResourceCache.installResource(InternalResourceCache.java:212)
	at com.oracle.truffle.polyglot.InternalResourceCache.installRuntimeResource(InternalResourceCache.java:173)
	... 85 common frames omitted

Overrridden assert fail:

Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.oracle.truffle.runtime.ModulesSupport.loadModulesSupportLibrary(ModulesSupport.java:164)
	... 134 more
Caused by: java.lang.AssertionError
	at com.oracle.truffle.polyglot.InternalResourceCache.installResource(InternalResourceCache.java:198)
	at com.oracle.truffle.polyglot.InternalResourceCache.installRuntimeResource(InternalResourceCache.java:173)
	... 139 more

Full example:

public class GraalTest {
    @Test
    void overridingResourcePath_succeeds_whenPropertyIsSet() {
        System.setProperty("polyglot.engine.resourcePath", System.getProperty("java.io.tmpdir"));
        Source.newBuilder("js", "const i = 1;", "Unnamed").buildLiteral();
    }
}

java.lang.InternalError: java.lang.reflect.InvocationTargetException

	at com.oracle.truffle.runtime.ModulesSupport.loadModulesSupportLibrary(ModulesSupport.java:171)
	at com.oracle.truffle.runtime.ModulesSupport.<clinit>(ModulesSupport.java:60)
	at com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntimeAccess.createRuntime(HotSpotTruffleRuntimeAccess.java:82)
	at com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntimeAccess.getRuntime(HotSpotTruffleRuntimeAccess.java:73)
	at com.oracle.truffle.api.Truffle.createRuntime(Truffle.java:145)
	at com.oracle.truffle.api.Truffle$1.run(Truffle.java:176)
	at com.oracle.truffle.api.Truffle$1.run(Truffle.java:174)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
	at com.oracle.truffle.api.Truffle.initRuntime(Truffle.java:174)
	at com.oracle.truffle.api.Truffle.<clinit>(Truffle.java:63)
	at com.oracle.truffle.api.impl.Accessor.getTVMCI(Accessor.java:1475)
	at com.oracle.truffle.api.impl.Accessor$Constants.<clinit>(Accessor.java:1340)
	at com.oracle.truffle.api.impl.Accessor.engineSupport(Accessor.java:1404)
	at com.oracle.truffle.api.library.LibraryAccessor.engineAccessor(LibraryAccessor.java:57)
	at com.oracle.truffle.api.library.LibraryFactory.loadExternalDefaultProviders(LibraryFactory.java:440)
	at com.oracle.truffle.api.library.LibraryFactory.getExternalDefaultProviders(LibraryFactory.java:431)
	at com.oracle.truffle.api.library.LibraryFactory.initDefaultExports(LibraryFactory.java:222)
	at com.oracle.truffle.api.library.LibraryFactory.<init>(LibraryFactory.java:217)
	at com.oracle.truffle.api.library.DynamicDispatchLibraryGen.<init>(DynamicDispatchLibraryGen.java:32)
	at com.oracle.truffle.api.library.DynamicDispatchLibraryGen.<clinit>(DynamicDispatchLibraryGen.java:24)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:467)
	at com.oracle.truffle.api.library.LibraryFactory.loadGeneratedClass(LibraryFactory.java:799)
	at com.oracle.truffle.api.library.LibraryFactory.resolveImpl(LibraryFactory.java:748)
	at com.oracle.truffle.api.library.LibraryFactory.resolve(LibraryFactory.java:741)
	at com.oracle.truffle.api.library.LibraryFactory.<init>(LibraryFactory.java:211)
	at com.oracle.truffle.api.interop.InteropLibraryGen.<init>(InteropLibraryGen.java:178)
	at com.oracle.truffle.api.interop.InteropLibraryGen.<clinit>(InteropLibraryGen.java:169)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:467)
	at com.oracle.truffle.api.library.LibraryFactory.loadGeneratedClass(LibraryFactory.java:799)
	at com.oracle.truffle.api.library.LibraryFactory.resolveImpl(LibraryFactory.java:748)
	at com.oracle.truffle.api.library.LibraryFactory.resolve(LibraryFactory.java:741)
	at com.oracle.truffle.api.interop.InteropLibrary.<clinit>(InteropLibrary.java:2941)
	at com.oracle.truffle.polyglot.PolyglotValueDispatch.<clinit>(PolyglotValueDispatch.java:167)
	at com.oracle.truffle.polyglot.PolyglotImpl.initialize(PolyglotImpl.java:199)
	at org.graalvm.polyglot.Engine.loadAndValidateProviders(Engine.java:1686)
	at org.graalvm.polyglot.Engine$1.run(Engine.java:1712)
	at org.graalvm.polyglot.Engine$1.run(Engine.java:1707)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
	at org.graalvm.polyglot.Engine.initEngineImpl(Engine.java:1707)
	at org.graalvm.polyglot.Engine$ImplHolder.<clinit>(Engine.java:190)
	at org.graalvm.polyglot.Engine.getImpl(Engine.java:442)
	at org.graalvm.polyglot.Source.getImpl(Source.java:140)
	at org.graalvm.polyglot.Source$Builder.build(Source.java:920)
	at org.graalvm.polyglot.Source$Builder.buildLiteral(Source.java:944)
	at x.y.z.js.GraalTest.overridingResourcePath_succeeds_whenPropertyIsSet(GraalTest.java:12)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
	at org.junit.jupiter.[xxxx]
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.oracle.truffle.runtime.ModulesSupport.loadModulesSupportLibrary(ModulesSupport.java:164)
	... 116 more
Caused by: java.lang.AssertionError
	at com.oracle.truffle.polyglot.InternalResourceCache.installResource(InternalResourceCache.java:198)
	at com.oracle.truffle.polyglot.InternalResourceCache.installRuntimeResource(InternalResourceCache.java:173)
	... 121 more


Process finished with exit code 255

[d-schmidt]

I tried to disable the cache on the sources. It doesn't help, the Engine loads the cache anyways.

[chumer]

@tzezula do you have a suggestion for a workaround? Whatever it is an assertion should not be reachable.

[d-schmidt]

I can't currently find a way to work around this. I can't change user.home for the app, I can't get write access, and I can't circumvent the test pipelines. Overriding the path in the current form doesn't work. There doesn't seem to be any flag to disable the cache completely.

I don't have further suggestions as I don't understand what "VERSIONED" does or allows. I would remove user.home or do whatever is necessary to have the overridden path create VERSIONED paths.

[tzezula]

The polyglot.engine.resourcePath system property needs to be used with an lready pre-created unversioned resources created by the Engine.html#copyResources method. It's not direct replacement for the versioned resources cache in the user.home. On Linux we support XDG_CACHE_HOME environemt variable that can be used to override the default cache folder ~/.cache for versioned resources.

The assertion error has been resolved in the commit 74bbd92 within GraalVM 24.0.0. I will backport this fix into GraalVM 23.1.3. Failing on assertion errors is not desirable. In GraalVM 24.0.0, the failure is now on a more reasonable error Can't load library: /Users/tom/Downloads/01/engine/libtruffleattach/bin/libtruffleattach.dylib.

will also enhance the Embedders Guide to provide a more detailed description of the polyglot.engine.resourcePath system property.

[d-schmidt]

The polyglot.engine.resourcePath system property needs to be used with an lready pre-created unversioned resources created by the Engine.html#copyResources method. It's not direct replacement for the versioned resources cache in the user.home.

This sounds like the current implementation is an unintended side-effect. Just from the diff I can't grasp how this solves the conflict of using polyglot.engine.resourcePath breaking the cache(test). I will wait and see. I will have the environment varaible checked

[tzezula]

When dealing with the polyglot.engine.resourcePath (unversioned resources), the code should avoid reaching that point and attempting to unpack any resources. The fix checks for this scenario. Unversioned resources have their path field set. Utilizing the polyglot.engine.resourcePath alone won't be effective unless an unversioned resource cache is precreated using the Engine#copyResources method. This method constructs a resources folder containing all necessary resources, which must be added to your Docker image. Subsequently, you can direct the polyglot.engine.resourcePath to this designated folder. I believe that using the XDG_CACHE_HOME environment variable might be simpler for your use case.

[tzezula]

I've introduced the polyglot.engine.userResourceCache system property, enabling users to customize the default versioned resources cache folder. This property serves as a cross-platform alternative to the XDG_CACHE_HOME environment variable. https://github.com/oracle/graal/commit/d85c8c960c0764cee4577c12d700691fdb3f8f2f