Task :metadataCopy fails with `FileNotFoundException`.

[tglaeser]

Description Executing Gradle task metadataCopy as outlined in Reflection support and running with the native agent fails with Error: Unknown name in option specification: macro:native-image-configure-launcher.

To Reproduce

$ ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image

> Task :metadataCopy FAILED
Error: Unknown name in option specification: macro:native-image-configure-launcher
Available macro options are:
    --macro:native-image-agent-library
    --macro:native-image-diagnostics-agent-library
    --macro:native-image-launcher

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':metadataCopy'.
> Process 'command '/opt/mandrel-java17-23.0.2.1-Final/bin/native-image'' finished with non-zero exit value 20

I'm not doing anything special, just following the above mentioned documentation.

System Info

$ uname -srmo
Linux 5.15.133.1-microsoft-standard-WSL2 x86_64 GNU/Linux

$ native-image --version
native-image 17.0.9 2023-10-17
GraalVM Runtime Environment Mandrel-23.0.2.1-Final (build 17.0.9+9)
Substrate VM Mandrel-23.0.2.1-Final (build 17.0.9+9, serial gc)

plugins {
	id 'java-library'
	id 'org.graalvm.buildtools.native' version '0.9.28'
}

Additional context I wonder if this is related to issue #194 and what the doc says here? If the Native Image Configure Tool is not guaranteed to be available and command

$ native-image --macro:native-image-configure-launcher
Error: Unknown name in option specification: macro:native-image-configure-launcher
Available macro options are:
    --macro:native-image-agent-library
    --macro:native-image-diagnostics-agent-library
    --macro:native-image-launcher

fails, how can the Gradle plugin depend on this functionality?

[tglaeser]

Anyone, any pointers please? I can successfully execute ./gradlew -Pagent run to collect the metadata, but ultimately I want to collect the metadata while executing the tests via ./gradlew -Pagent nativeTest which depends on the above mentioned task. This is critical functionality to me. What am I missing?

[tglaeser]

@melix, @dnestoro - Any pointers from you please? Thanks.

[tglaeser]

Anything wrong with this ticket/question?

[fniephaus]

Hi @tglaeser, sorry for the delayed response. It seems you are using Mandrel, which is a GraalVM distro specifically for Quarkus. Unfortunately, Quarkus is not using the Native Build Tools, and apparently, Mandrel is not tested with them. Could you please try this again with GraalVM CE or Oracle GraalVM?

[tglaeser]

Yes, that makes a difference. Command sequence

$ ./gradlew -Pagent clean test
$ ./gradlew metadataCopy --task test --dir src/main/resources/META-INF/native-image/

works. However

$ ./gradlew -Pagent clean run
$ ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image/

continues to fail, now with error

> Task :metadataCopy FAILED
java.io.FileNotFoundException: .../build/native/agent-output/run/reflect-config.json (No such file or directory)

This is is because the metadata has been written under .../build/native/agent-output/run/session-28305-20240306T222136Z, not .../build/native/agent-output/run.

[fniephaus]

This is is because the metadata has been written under .../build/native/agent-output/run/session-28305-20240306T222136Z, not .../build/native/agent-output/run.

@dnestoro could you please try reproduce and fix this?

[dnestoro]

@dnestoro could you please try reproduce and fix this?

Sure! I will take a look today or tomorrow

[melix]

The launcher is uses to build the tool responsible for merging agent files. If there's no launcher, we cannot merge these.

[tglaeser]

Now that I got your attention, I'm seeing other warnings/NPE's when executing a task with the agent attached:

> Task :test
[0.080s][warning][jni,resolve] Re-registering of platform native method: jdk.internal.misc.Unsafe.allocateInstance(Ljava/lang/Class;)Ljava/lang/Object; from code in a different classloader

Warning: Error processing trace entry map(size=5, {(tracer,reflect),(function,getSystemResource),(caller_class,org.apache.logging.log4j.core.util.Loader),(result,true),(args,[null])}): java.lang.NullPointerException
        at [email protected]/java.util.regex.Pattern.quote(Pattern.java:1499)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.trace.ReflectionProcessor.processEntry(ReflectionProcessor.java:85)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.trace.TraceProcessor.processEntry(TraceProcessor.java:88)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.tracing.ConfigurationResultWriter.traceEntry(ConfigurationResultWriter.java:53)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.tracing.core.Tracer.traceCall(Tracer.java:117)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.traceBreakpoint(BreakpointInterceptor.java:187)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.traceReflectBreakpoint(BreakpointInterceptor.java:176)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.handleGetSystemResources(BreakpointInterceptor.java:619)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.getSystemResource(BreakpointInterceptor.java:609)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.onBreakpoint(BreakpointInterceptor.java:1112)

Do you want me to open another ticket for that?

[fniephaus]

Yes, please. Could you open that at https://github.com/oracle/graal/issues?

[fniephaus]

The launcher is uses to build the tool responsible for merging agent files. If there's no launcher, we cannot merge these.

We should probably fail with a better error message in the first place if native-image-configure cannot be found.

This is is because the metadata has been written under .../build/native/agent-output/run/session-28305-20240306T222136Z, not .../build/native/agent-output/run.

@melix this is probably another issue that we should fix.

[melix]

I think it's the same issue. The session-28305-20240306T222136Z directories are created by during the execution of the agent. The reason there is a timestamp is because you can typically have multiple JVMs forked during testing, and each needs to write into its own directory. Then the files are merged, and I think we have this directory kept only because the merge failed.

[tglaeser]

Now, that's interesting. Above I said the Gradle task sequence is

$ ./gradlew -Pagent clean run
$ ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image/

More specifically, after executing the run task I'm hitting Ctrl+C to stop the task. This simply might not be graciously enough to allow the merge action to be executed.

[melix]

mmm, indeed this may be an issue. I don't know what kind of app you are building but maybe adding an endpoint to shutdown the app gracefully would help?

[tglaeser]

Just verified that: Gracefully shutting down the application solves the issue.

[tglaeser]

The issue is understood, closing the ticket.