Prototype implementation for #36822

Here are the implementation details:

It adds a new profile, called native-with-agent to the generated projects where the JVM mode tests run with the native image agent. This profile sets the native image agent parameter necessary to run the JVM mode tests with. This parameter includes references to filters that are produced before running JVM mode tests.

Before the JVM mode tests run, GenerateCodeTestsMojo has been modified to generated the native image agent filters are produced. This enables filtering out most of the configuration that is quarkus specific, but crucially it doesn't handle resources. One caveat of this early filtering is that we don't have context of the application to do more fine grained filtering. An alternative implementation would skip filtering at this level and instead filter at the build step phase, just like resources are filtered (read on).

During JVM mode tests, native image agent generates config files inside target/generated-native-config.

Then, native image agent specific build steps pick up this configuration, parse it, filtering using regex and produce existing build items, e.g. ReflectiveClassBuildItem, NativeImageResourcePatternsBuildItem...etc.

There is a builder class that produces json but there's no class to read json files, so I've gone ahead and created a small, dependency-free, json parser called JsonReader for this. The objective of this parser was to get something working quickly and without dependencies, so I'm sure there's a lot of ways in which it can be improved :)

As mentioned above, resources cannot be filtered at the native image agent level, so I post-processed the resources configuration file using the JsonReader parser, along with regex, to filter out resources that Quarkus takes care of.

One important note here is that both the native image configuration that Quarkus produces, and the one extracted from the native image agent ends up in the same configuration files inside the generated bytecode. Achieving this is quite simple because we just reuse the same existing build items. But it might make debugging easier if the files were different so that we can distinguish between those Quarkus produces and those that come from the native image agent more easily. Not sure this is doable, @zakkak? Otherwise, you can always run the app with -Dnative and -Dnative-with-agent and compare the json files (see a demonstration below).

I've tested out this in a new project generated from this branch called new-project (attached new-project-0.1.tar.gz). This is a small example that requires reflection to be configured for Alice class lookup in order to respond to the /hello/Alice endpoint. The native image agent integration works when the native integration test for that endpoint is a success. Comparing the output of the following commands with -Dnative and -Dnative-with-agent you can see what impact the native image agent jvm mode test integration has on the configuration:

$ ./mvnw verify -Dnative 
...
$ tar xfO ./target/quarkus-app/quarkus/generated-bytecode.jar META-INF/native-image/reflect-config.json | jq -s '.[] |= sort_by(.name)' > target/reflect-config-no-agent.json
...
$ ./mvnw verify -Dnative-with-agent
...
$ tar xfO ./target/quarkus-app/quarkus/generated-bytecode.jar META-INF/native-image/reflect-config.json | jq -s '.[] |= sort_by(.name)' > target/reflect-config-with-agent.json
...
$ diff target/reflect-config-with-agent.json target/reflect-config-no-agent.json
619,628d618
<     },
<     {
<       "methods": [
<         {
<           "parameterTypes": [],
<           "name": "sayMyName"
<         }
<       ],
<       "allDeclaredConstructors": true,
<       "name": "org.acme.Alice"
632,635d621
<       "name": "org.acme.Bob"
<     },
<     {
<       "allDeclaredConstructors": true,

It has added Alice class and its sayMyName method to the reflection configuration, which is required in order for endpoint to respond.

There are also differences in the resource configuration but they are not relevant here:

$ tar xfO ./new-project/target/quarkus-app/quarkus/generated-bytecode.jar META-INF/native-image/resource-config.json | jq -s '.[].resources.includes |= sort_by(.pattern)' ...
...
$ diff with-agent/resource-config-with-agent.json no-agent/resource-config-no-agent.json
31,78d30
<         },
<         {
<           "pattern": "\\\\QMETA-INF/maven/org\\.apache\\.maven/maven-core/pom\\.properties\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/plexus/components\\.xml\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/java\\.lang\\.System\\$LoggerFinder\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/java\\.net\\.spi\\.InetAddressResolverProvider\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/java\\.net\\.spi\\.URLStreamHandlerProvider\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/java\\.nio\\.channels\\.spi\\.SelectorProvider\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/java\\.nio\\.file\\.spi\\.FileSystemProvider\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/java\\.time\\.zone\\.ZoneRulesProvider\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/javax\\.xml\\.stream\\.XMLInputFactory\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/services/org\\.codehaus\\.groovy\\.runtime\\.ExtensionModule\\\\E"
<         },
<         {
<           "pattern": "\\\\QMETA-INF/sisu/javax\\.inject\\.Named\\\\E"
<         },
<         {
<           "pattern": "\\\\Qjava/io/Serializable\\.class\\\\E"
<         },
<         {
<           "pattern": "\\\\Qorg/apache/maven/model/pom-4\\.0\\.0\\.xml\\\\E"
<         },
<         {
<           "pattern": "\\\\Qorg/osgi/annotation/bundle/Requirements\\.class\\\\E"
<         },
<         {
<           "pattern": "java\\.base:\\\\Qjava/io/Serializable\\.class\\\\E"
<         },
<         {
<           "pattern": "java\\.base:\\\\Qjdk/internal/icu/impl/data/icudt72b/nfc\\.nrm\\\\E"

One thing of interest in the resource differences is that it uncovers a bug in the native image integration. It's escaping patterns when it shouldn't, see #36823.

Final note, what to filter for is up for debate. I was quite aggressive in the native image agent filters because there really wasn't need for any of the additional configuration that it generated, but certain filters are debatable. Not sure what is the best approach here. Do we prefer minimal filtering that we now for sure Quarkus takes care of? Or do we prefer aggressive filtering? I think I would probably go for minimal because the aim here is to get something that native image builds and works. The user can trim things later if necessary.

Nov 02 '23 08:11 galderz

But it might make debugging easier if the files were different so that we can distinguish between those Quarkus produces and those that come from the native image agent more easily. Not sure this is doable, @zakkak? Otherwise, you can always run the app with -Dnative and -Dnative-with-agent and compare the json files (see a demonstration below).

@galderz that's definitely possible, but we would need to explicitly pass these config files to native-image using -H:ReflectionConfigurationFiles, -H:ResourceConfigurationFiles, -H:JNIConfigurationFiles, and -H:SerializationConfigurationFiles which are considered "experimental".

I wonder if we could add some kind of comment in the build items and output it in the json files, in the future we could augment the proposed reasons field.

Either way would be acceptable I believe.

Nov 02 '23 10:11 zakkak

Do we prefer minimal filtering that we now for sure Quarkus takes care of? Or do we prefer aggressive filtering? I think I would probably go for minimal because the aim here is to get something that native image builds and works. The user can trim things later if necessary.

Minimal sounds good, as long as the user can include and edit the generated parts in their source code repository. Things might become tricky though when later you do an update and rerun native-with-agent since you will have to do the clean up again.

Nov 02 '23 10:11 zakkak

don't we control the exeuction of tests and run enough to run with different agents? feel like i recall @stuartwdouglas or @geoand mention we could register agents?

that could avoid need of manipulating pom.xml/build.gradle?

Nov 02 '23 14:11 maxandersen

@maxandersen we can fully control the launch of @QuarkusIntegrationTest. The JVM used for @QuarkusTest (and all "unit" tests for that matter) is controlled by the surefire plugin in Maven.

Nov 02 '23 14:11 geoand

Do we prefer minimal filtering that we now for sure Quarkus takes care of? Or do we prefer aggressive filtering? I think I would probably go for minimal because the aim here is to get something that native image builds and works. The user can trim things later if necessary.

Minimal sounds good, as long as the user can include and edit the generated parts in their source code repository. Things might become tricky though when later you do an update and rerun native-with-agent since you will have to do the clean up again.

This got me thinking.

I think the best way to use this integration would be to run it locally, with GraalVM installed locally, take the generated config and add it to your source tree, as you already suggest above. The main reason for doing this is because we would not be able to execute native-with-agent profile with in-container builds.

If the user then wants to add additional configuration manually via JSON, it would work best if the files were differently named to the ones that were added to the source tree as a result of native-image-agent execution. That would make it easy to re-run native-with-agent and replace those files.

So, going back to your other comment about different files vs reasons, I think different files would work better given the above.

Nov 06 '23 09:11 galderz

Thanks all for the feedback. I've had a quick chat with @zakkak to clarify a few things. I'll work on the next iteration of the feature and I'll post an update when ready.

Nov 09 '23 08:11 galderz

I've just pushed a new prototype that works like this:

The maven project generation still adds a new profile native-with-agent and that makes surefire unit tests run with native image agent.
The native image agent spits out the configuration into target/native-agent-base-config folder for maven.
There is no longer filtering done at the agent level and instead all the configuration filtering is delegated to the quarkus build inside NativeImageAgentConfigStep.
NativeImageAgentConfigStep reads the native image agent json using the new JsonReader class and that is then transformed using the new JsonTransform interface, combined with the existing Json class that emits json (The transformation API is inspired by how the upcoming Class-File API in OpenJDK works). So, there are no longer intermediate build items created. It just takes json in, transforms it and spits it out onto the folder <source-jar-dir>/native-agent-config folder for maven.
The JsonTransform API only has one function, which is to drop json from the input. That's all that is needed for this use case, but more functions could be added for future use cases.
The Json API has been enhanced to add a new skipEscape option to avoid escaping strings where they are already escape. That's the case with transformation where the input json is already escaped and hence does not need to be escaped again.
I've created different regular expression filters for each configuration file. The regular expressions are simple: if there is a reflection class FQN containing the word quarkus, it's filtered out...etc. For each configuration file type, there's a different lambda that figures out at which level the filtering needs to happen.
The contents of native-agent-config are hooked into native image via -H:ConfigurationFileDirectories= option. @zakkak Looking at the source code this option is stable (iow not experimental), so seems like a better option that individual file configuration parameters.
integration-tests/maven/src/test/java/io/quarkus/maven/it/NativeAgentIT has been added that verifies that native image integration unit tests work for a project that needs reflection configuration to succeed. See associated project code.
integration-tests/gradle/src/test/java/io/quarkus/gradle/nativeimage/NativeAgentIntegrationTestIT.java has been added that verifies the same thing but gradle projects.

I have the following questions, doubts or comments:

The maven project unit test bakes the new profile that would be seen by new projects generated which is not ideal, but I don't see tests that generate maven projects on the file. Any thoughts on this?
The Gradle project is in integration-tests/gradle/src/main/resources/it-test-native-agent-project and there I manually added the test jvmArgs to contain the native image agent parameters (note that the folder where it spits it out is instead build/native-agent-base-config). I do see gradle project generation is in quarkus, but how would you add a profile (ala -Dnative-with-agent in maven) in Gradle so that those jvmArgs are only added when the given profile/option is active?
Seems to me the generic type in Json.JsonBuilder adds unnecessary complexity. Its self method, which is the only one that uses the generic is not used at all.
It probably makes sense to have a single regular expression that works for all configuration file types, because in the end what is filtered is common. This avoids duplication of keywords to match. It is also true that sometimes you're filtering FQN classes and other times resource paths and so the point could be made that finer tuned regular expressions would result in different regex for different configuration files, but seems to me that would makes regex overcomplicated.
I explored adding quarkus:run integration for maven and that might be doable, but the equivalent feature for Gradle needs further work (see https://github.com/quarkusio/quarkus/issues/37379), so I think it's better to defer that to a later stage.

Nov 29 '23 10:11 galderz

when you are generating the missing declarations, it would be nice to display it somewhere readable. for instance in the build logs. this would allow the developer to figure out what he missed, and give him the option to add the declaration explicitly in the app.

Nov 30 '23 07:11 vsevel

Great work, @galderz. I like the new approach even better :)

Dec 01 '23 08:12 zakkak

when you are generating the missing declarations, it would be nice to display it somewhere readable. for instance in the build logs. this would allow the developer to figure out what he missed, and give him the option to add the declaration explicitly in the app.

Fair enough. I could add some log messages that specify the folders where the native image agent generated config files and the transformed ones are located. Does that sound good?

Dec 04 '23 16:12 galderz

I could add some log messages that specify the folders where the native image agent generated config files and the transformed ones are located.

ideally I would give more control to the developer. at least in our env, dev teams do not have access to the CI workspace. they have to go through the CI team to get an extract. I would rather print a log message saying that 26 classes have been added to the reflection json file, and offer to specify an option -DdisplayNativeAgentFileContent=true to show the content. unless you think this could be huge (we could even add a quality gate searching for this pattern). ideally I would like the dev team to be autonomous. I suspect the CI team is going to be bored pretty quickly. and I suspect they will not want to provide access to the nodes.

Dec 04 '23 17:12 vsevel

I could add some log messages that specify the folders where the native image agent generated config files and the transformed ones are located.

ideally I would give more control to the developer. at least in our env, dev teams do not have access to the CI workspace. they have to go through the CI team to get an extract. I would rather print a log message saying that 26 classes have been added to the reflection json file, and offer to specify an option -DdisplayNativeAgentFileContent=true to show the content. unless you think this could be huge (we could even add a quality gate searching for this pattern). ideally I would like the dev team to be autonomous. I suspect the CI team is going to be bored pretty quickly. and I suspect they will not want to provide access to the nodes.

Ok, I understand the situation now. I think an option to show native image agent configuration files in the console would make sense to cover such scenario, but it would work best if only the post-processed configuration would be shown. So, not what comes out directly out of the native image agent, but after having that filtered out to remove things that Quarkus takes care of.

For the most basic of Quarkus applications that has a just returns a greeting message, these are the sizes of files generated by native image agent:

$ l ./new-project/target/native-agent-base-config
total 872
drwxr-xr-x   9 galder  staff   288B  7 Dec 09:45 .
drwxr-xr-x  18 galder  staff   576B  7 Dec 09:46 ..
-rw-r--r--   1 galder  staff   967B  7 Dec 09:45 jni-config.json
-rw-r--r--   1 galder  staff    86B  7 Dec 09:45 proxy-config.json
-rw-r--r--   1 galder  staff   406K  7 Dec 09:45 reflect-config.json
-rw-r--r--   1 galder  staff   8.6K  7 Dec 09:45 resource-config.json
-rw-r--r--   1 galder  staff   2.1K  7 Dec 09:45 serialization-config.json

The biggest file there is by quite a big marging reflect-config.json with 406K. Once post-processed I see:

$ l ./new-project/target/new-project-1.0.0-SNAPSHOT-native-image-source-jar/native-agent-config
total 152
drwxr-xr-x  7 galder  staff   224B  7 Dec 09:45 .
drwxr-xr-x  6 galder  staff   192B  7 Dec 09:46 ..
-rw-r--r--  1 galder  staff   785B  7 Dec 09:45 jni-config.json
-rw-r--r--  1 galder  staff     2B  7 Dec 09:45 proxy-config.json
-rw-r--r--  1 galder  staff    55K  7 Dec 09:45 reflect-config.json
-rw-r--r--  1 galder  staff   4.8K  7 Dec 09:45 resource-config.json
-rw-r--r--  1 galder  staff   767B  7 Dec 09:45 serialization-config.json

The reflection configuration has dropped to 55K. Technically this could be further brought down but it's not easy for Quarkus to know exactly, because these remaining registrations are all related to java, javax and sun packages, so the application itself could have caused those to be required. So, the post-processing is quite conservative here, but if printed out they can always be added to the source code and do further filtering there if needed.

Dec 07 '23 09:12 galderz

So, not what comes out directly out of the native image agent, but after having that filtered out to remove things that Quarkus takes care of.

yes

but if printed out they can always be added to the source code and do further filtering there if needed.

yes exactly having the agent to detect something means that the developer has forgotten declaring some resources in the app. and we are down to having the tests to identify the missing pieces. if a test is disabled at some point, we could forget declaring some resources, and get a regression on the native build. it is kind of awkward if you think about it: activating or deactivating a test now has an impact on the built app, whereas tests should only observe and report.

Dec 07 '23 10:12 vsevel

if/when your reporting feature is available, I would instruct the teams to review the generated content systematically, as it is smell to detect something. I think this should be logged as a WARN

Dec 07 '23 10:12 vsevel

I had a follow up chat with @aloubyansky last week and I'm working on a further iteration of this. I'm also including the feedback that @vsevel provided. I will update once I have something to show once again.

Dec 18 '23 16:12 galderz

I've just pushed some updates, here are the relevant changes:

The native image agent integration impact is now informative rather than affecting the native executable build by default. This addresses potential issues raised by @vsevel whereby ignoring tests could lead to different native image outcomes. An example of the output would look something like this:

[INFO] [io.quarkus.deployment.steps.NativeImageAgentConfigStep] Discovered native image agent generated files
[INFO] [io.quarkus.deployment.steps.NativeImageAgentConfigStep] Generated reflect-config.json:
[
{
  "name":"org.acme.Alice",
  "methods":[{"name":"<init>","parameterTypes":[] }, {"name":"sayMyName","parameterTypes":[] }]
},
{
  "name":"org.acme.Bob"
}
]

This output currently appears as part of the native image process. I could potentially print this once unit tests has completed (it would probably a better time for it), but couldn't see how to adjust a mojo to do such a thing. @aloubyansky any idea?

I added an option to the native agent called agentConfigurationApply. This option is disabled by default, but when enabled it applies the native image agent generated and post-processed configuration into the current native executable process. This can be used by users to skip the step of copying the generated configuration that is printed in the log to separate files and you can use it right away. However, this should be done with caution for the reasons explained above. I'll expand the javadoc option with some warning advice.
I've brought back generation of access and caller json filters for the native image agent. With the guidance of @aloubyansky I've added this to a new GenerateNativeImageAgentFiltersMojo class that consumes the application model to construct the filters. The class adds filters to skip packages located in dependencies that either test scope, or deployment classpath only (IOW, not in runtime classpath). It then takes all these packages to skip and shrinks them to 3 component packages (e.g. a.b.c). The number 3 is a bit arbitrary but seemed to be enough to filter undesired packages for now. We could up this to say 4/5 package components if needed.
Resource configuration is not handled by the native image agent access or caller filters, so post-processing for this file remains. This filtering here is still based on hardcoded regular expressions, but I will try to expand this filtering further to skip resources in test or deployment classpath only.
Filtering at the native image level level and post-processing for resource configuration has very positive impact reducing the configuration, but some very small outlayers remain due to caller information not present and since they are all related to JDK classes, I don't think I can filter those out 100%. Full output of a integration test added in this PR can be found here.
Note that the configuration output is nicely formatted for all but resource configuration. This is due to post-processing happening for resource configuration, which is handled by the JSON generator in the builder project. I'll look into adding some formatting there.

Jan 08 '24 17:01 galderz

I could potentially print this once unit tests has completed (it would probably a better time for it), but couldn't see how to adjust a mojo to do such a thing. @aloubyansky any idea?

Sounds like it would have to be another goal that would be executed after the tests.

Jan 31 '24 13:01 aloubyansky

@aloubyansky @zakkak @maxandersen @vsevel This PR is not a draft any more. I've updated the description of #36822, which explains in detail the motivation, how the integration works and how to debug it. Parts of it will become the documentation once consensus is achieved.

Feb 22 '24 09:02 galderz

:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit 02c1b3c83e7c897b4647b66cd1749d1359a0a5cd.

Failing Jobs

Status	Name	Step	Failures	Logs	Raw logs	Build scan
✖	JVM Tests - JDK 17	`Build`	Failures	Logs	Raw logs	:construction:
✖	JVM Tests - JDK 17 Windows	`Build`	Failures	Logs	Raw logs	:construction:
✖	JVM Tests - JDK 21	`Build`	Failures	Logs	Raw logs	:construction:
✖	Maven Tests - JDK 17	`Build`	Failures	Logs	Raw logs	:construction:
✖	Maven Tests - JDK 17 Windows	`Build`	Failures	Logs	Raw logs	:construction:
✖	Native Tests - Data5	`Build`	Failures	Logs	Raw logs	:construction:
✖	Native Tests - Misc1	`Build`	Failures	Logs	Raw logs	:construction:
✖	Native Tests - Misc4	`Build`	Failures	Logs	Raw logs	:construction:

Full information is available in the Build summary check run.

Failures

:gear: JVM Tests - JDK 17 #

- Failing: independent-projects/tools/devtools-testing 
! Skipped: devtools/cli devtools/gradle/gradle-application-plugin devtools/gradle/gradle-extension-plugin and 7 more

:package: independent-projects/tools/devtools-testing

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartGenerationTest.generateDefault(TestInfo) line 57 - History - More details - Source on GitHub

java.lang.AssertionError: 
[Snapshot is not matching (use -Dsnap to update it automatically): QuarkusCodestartGenerationTest/generateDefault/pom.xml] 
Path:
  target/quarkus-codestart-gen-test/default/pom.xml
and path:
  /home/runner/work/quarkus/quarkus/independent-projects/tools/devtools-testing/target/test-classes/__snapshots__/QuarkusCodestartGenerationTest/generateDefault/pom.xml
do not have same content:

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartGenerationTest.generateMavenWithCustomDep(TestInfo) line 115 - History - More details - Source on GitHub

java.lang.AssertionError: 
[Snapshot is not matching (use -Dsnap to update it automatically): QuarkusCodestartGenerationTest/generateMavenWithCustomDep/pom.xml] 
Path:
  target/quarkus-codestart-gen-test/maven-custom-dep/pom.xml
and path:
  /home/runner/work/quarkus/quarkus/independent-projects/tools/devtools-testing/target/test-classes/__snapshots__/QuarkusCodestartGenerationTest/generateMavenWithCustomDep/pom.xml
do not have same content:

:gear: JVM Tests - JDK 17 Windows #

- Failing: independent-projects/tools/devtools-testing 
! Skipped: devtools/cli devtools/gradle/gradle-application-plugin devtools/gradle/gradle-extension-plugin and 7 more

:package: independent-projects/tools/devtools-testing

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartGenerationTest.generateDefault(TestInfo) line 57 - History - More details - Source on GitHub

java.lang.AssertionError: 
[Snapshot is not matching (use -Dsnap to update it automatically): QuarkusCodestartGenerationTest/generateDefault/pom.xml] 
Path:
  target\quarkus-codestart-gen-test\default\pom.xml
and path:
  D:\a\quarkus\quarkus\independent-projects\tools\devtools-testing\target\test-classes\__snapshots__\QuarkusCodestartGenerationTest\generateDefault\pom.xml
do not have same content:

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartGenerationTest.generateMavenWithCustomDep(TestInfo) line 115 - History - More details - Source on GitHub

java.lang.AssertionError: 
[Snapshot is not matching (use -Dsnap to update it automatically): QuarkusCodestartGenerationTest/generateMavenWithCustomDep/pom.xml] 
Path:
  target\quarkus-codestart-gen-test\maven-custom-dep\pom.xml
and path:
  D:\a\quarkus\quarkus\independent-projects\tools\devtools-testing\target\test-classes\__snapshots__\QuarkusCodestartGenerationTest\generateMavenWithCustomDep\pom.xml
do not have same content:

:gear: JVM Tests - JDK 21 #

- Failing: independent-projects/tools/devtools-testing 
! Skipped: devtools/cli devtools/gradle/gradle-application-plugin devtools/gradle/gradle-extension-plugin and 7 more

:package: independent-projects/tools/devtools-testing

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartGenerationTest.generateDefault(TestInfo) line 57 - History - More details - Source on GitHub

java.lang.AssertionError: 
[Snapshot is not matching (use -Dsnap to update it automatically): QuarkusCodestartGenerationTest/generateDefault/pom.xml] 
Path:
  target/quarkus-codestart-gen-test/default/pom.xml
and path:
  /home/runner/work/quarkus/quarkus/independent-projects/tools/devtools-testing/target/test-classes/__snapshots__/QuarkusCodestartGenerationTest/generateDefault/pom.xml
do not have same content:

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartGenerationTest.generateMavenWithCustomDep(TestInfo) line 115 - History - More details - Source on GitHub

java.lang.AssertionError: 
[Snapshot is not matching (use -Dsnap to update it automatically): QuarkusCodestartGenerationTest/generateMavenWithCustomDep/pom.xml] 
Path:
  target/quarkus-codestart-gen-test/maven-custom-dep/pom.xml
and path:
  /home/runner/work/quarkus/quarkus/independent-projects/tools/devtools-testing/target/test-classes/__snapshots__/QuarkusCodestartGenerationTest/generateMavenWithCustomDep/pom.xml
do not have same content:

:gear: Maven Tests - JDK 17 #

- Failing: integration-tests/maven

:package: integration-tests/maven

✖ io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch line 107 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch(CreateProjectMojoIT.java:107)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

✖ io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch line 107 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch(CreateProjectMojoIT.java:107)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

:gear: Maven Tests - JDK 17 Windows #

- Failing: integration-tests/maven

:package: integration-tests/maven

✖ io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch line 107 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch(CreateProjectMojoIT.java:107)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

✖ io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch line 107 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch(CreateProjectMojoIT.java:107)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

:gear: Native Tests - Data5 #

- Failing: integration-tests/jpa-postgresql-withxml

:package: integration-tests/jpa-postgresql-withxml

✖ io.quarkus.it.jpa.postgresql.ImageMetricsITCase.verifyImageMetrics line 15 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: Expected analysis_results.fields.reflection to be within range [163 +- 3%] but was 168 ==> expected: <true> but was: <false>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
	at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
	at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
	at io.quarkus.test.junit.nativeimage.NativeBuildOutputExtension.assertValueWithinRange(NativeBuildOutputExtension.java:90)
	at io.quarkus.test.junit.nativeimage.NativeBuildOutputExtension.lambda$verifyImageMetrics$0(NativeBuildOutputExtension.java:66)

:gear: Native Tests - Misc1 #

- Failing: integration-tests/maven

:package: integration-tests/maven

✖ io.quarkus.maven.it.NativeAgentIT.testRunIntegrationTests line 25 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0
 but was: 1
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)

:gear: Native Tests - Misc4 #

- Failing: integration-tests/gradle

:package: integration-tests/gradle

✖ io.quarkus.gradle.nativeimage.NativeAgentIntegrationTestIT.nativeTestShouldRunIntegrationTest line 22 - History - More details - Source on GitHub

java.lang.AssertionError: Gradle build failed with exit code 1
	at io.quarkus.gradle.QuarkusGradleWrapperTestBase.runGradleWrapper(QuarkusGradleWrapperTestBase.java:140)
	at io.quarkus.gradle.QuarkusGradleWrapperTestBase.runGradleWrapper(QuarkusGradleWrapperTestBase.java:57)
	at io.quarkus.gradle.QuarkusGradleWrapperTestBase.runGradleWrapper(QuarkusGradleWrapperTestBase.java:52)
	at io.quarkus.gradle.nativeimage.QuarkusNativeGradleITBase.runGradleWrapper(QuarkusNativeGradleITBase.java:36)
	at io.quarkus.gradle.nativeimage.NativeAgentIntegrationTestIT.nativeTestShouldRunIntegrationTest(NativeAgentIntegrationTestIT.java:22)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Flaky tests - Develocity

:gear: JVM Tests - JDK 21

:package: extensions/smallrye-reactive-messaging-kafka/deployment

✖ io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

Assertion condition Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: 
Assertion condition 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

Feb 22 '24 14:02 quarkus-bot[bot]

@aloubyansky @zakkak @maxandersen @vsevel This PR is not a draft any more. I've updated the description of #36822, which explains in detail the motivation, how the integration works and how to debug it. Parts of it will become the documentation once consensus is achieved.

Hi @galderz that is my attention to spend some time on it

Feb 22 '24 17:02 vsevel

Looking into the test failures.

Feb 26 '24 05:02 galderz

:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit 4916fd264c46d2f253d14cea2a589104ecb89004.

Failing Jobs

Status	Name	Step	Failures	Logs	Raw logs	Build scan
✖	JVM Tests - JDK 17	`Build`	Failures	Logs	Raw logs	:construction:
✖	JVM Tests - JDK 17 Windows	`Build`	Failures	Logs	Raw logs	:construction:
✖	JVM Tests - JDK 21	`Build`	Failures	Logs	Raw logs	:construction:
✖	Native Tests - Data5	`Build`	Failures	Logs	Raw logs	:construction:

Full information is available in the Build summary check run.

Failures

:gear: JVM Tests - JDK 17 #

- Failing: integration-tests/kotlin integration-tests/narayana-lra integration-tests/scala

:package: integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin line 74 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin(KotlinCreateMavenProjectIT.java:74)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

:package: integration-tests/narayana-lra

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRA line 63 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <201> but was: <503>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
	at org.acme.quickstart.lra.LRAParticipantTest.testLRA(LRAParticipantTest.java:63)

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd line 44 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <201> but was: <500>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
	at org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd(LRAParticipantTest.java:44)

:package: integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaCreateMavenProjectIT.testProjectGenerationFromScratchForScala line 75 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.scala.maven.it.ScalaCreateMavenProjectIT.testProjectGenerationFromScratchForScala(ScalaCreateMavenProjectIT.java:75)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

:gear: JVM Tests - JDK 17 Windows #

- Failing: integration-tests/kotlin integration-tests/scala

:package: integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin line 74 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin(KotlinCreateMavenProjectIT.java:74)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

:package: integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaCreateMavenProjectIT.testProjectGenerationFromScratchForScala line 75 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.scala.maven.it.ScalaCreateMavenProjectIT.testProjectGenerationFromScratchForScala(ScalaCreateMavenProjectIT.java:75)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

:gear: JVM Tests - JDK 21 #

- Failing: integration-tests/kotlin integration-tests/narayana-lra

:package: integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin line 74 - History - More details - Source on GitHub

java.lang.AssertionError: 

Expected size: 1 but was: 2 in:
[Profile {id: native, source: pom},
    Profile {id: native-with-agent, source: pom}]
	at io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin(KotlinCreateMavenProjectIT.java:74)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

:package: integration-tests/narayana-lra

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRA line 63 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <201> but was: <503>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
	at org.acme.quickstart.lra.LRAParticipantTest.testLRA(LRAParticipantTest.java:63)

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd line 44 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <201> but was: <500>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
	at org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd(LRAParticipantTest.java:44)

:gear: Native Tests - Data5 #

- Failing: integration-tests/jpa-postgresql-withxml

:package: integration-tests/jpa-postgresql-withxml

✖ io.quarkus.it.jpa.postgresql.ImageMetricsITCase.verifyImageMetrics line 15 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: Expected analysis_results.fields.reflection to be within range [163 +- 3%] but was 168 ==> expected: <true> but was: <false>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
	at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
	at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
	at io.quarkus.test.junit.nativeimage.NativeBuildOutputExtension.assertValueWithinRange(NativeBuildOutputExtension.java:90)
	at io.quarkus.test.junit.nativeimage.NativeBuildOutputExtension.lambda$verifyImageMetrics$0(NativeBuildOutputExtension.java:66)

Flaky tests - Develocity

:gear: JVM Tests - JDK 21

:package: integration-tests/opentelemetry

✖ io.quarkus.it.opentelemetry.EndUserEnabledTest.baseTest - History

AttributesMap{data={http.route=/otel/enduser, code.namespace=io.quarkus.it.opentelemetry.util.EndUserResource, http.response_content_length=0, http.scheme=http, http.method=GET, net.host.port=8081, http.status_code=200, net.protocol.name=http, http.target=/otel/enduser, code.function=dummy, net.host.name=localhost, user_agent.original=Apache-HttpClient/4.5.14 (Java/21.0.2), http.client_ip=127.0.0.1}, capacity=128, totalAddedValues=13} ==> expected: <testUser> but was: <null> - org.opentest4j.AssertionFailedError

org.opentest4j.AssertionFailedError: AttributesMap{data={http.route=/otel/enduser, code.namespace=io.quarkus.it.opentelemetry.util.EndUserResource, http.response_content_length=0, http.scheme=http, http.method=GET, net.host.port=8081, http.status_code=200, net.protocol.name=http, http.target=/otel/enduser, code.function=dummy, net.host.name=localhost, user_agent.original=Apache-HttpClient/4.5.14 (Java/21.0.2), http.client_ip=127.0.0.1}, capacity=128, totalAddedValues=13} ==> expected: <testUser> but was: <null>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1156)
	at io.quarkus.it.opentelemetry.EndUserEnabledTest.evaluateAttributes(...

:gear: Maven Tests - JDK 17

:package: integration-tests/maven

✖ io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed - History

io.quarkus.maven.it.DevMojoIT expected "a67fc8a9-f939-4e19-b133-2d2a35d6cef1" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: io.quarkus.maven.it.DevMojoIT expected "a67fc8a9-f939-4e19-b133-2d2a35d6cef1" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AbstractHamcrestCondition.await(AbstractHamcrestCondition.java:86)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:691)
	at io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed(DevMojoIT.java:967)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

✖ io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed - History

io.quarkus.maven.it.DevMojoIT expected "a67fc8a9-f939-4e19-b133-2d2a35d6cef1" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: io.quarkus.maven.it.DevMojoIT expected "a67fc8a9-f939-4e19-b133-2d2a35d6cef1" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AbstractHamcrestCondition.await(AbstractHamcrestCondition.java:86)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:691)
	at io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed(DevMojoIT.java:967)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Feb 26 '24 13:02 quarkus-bot[bot]

:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit 41c1abca63703f628ba313bc5827d2bb0363ab14.

:warning: Unable to include the stracktraces as the report was too long. See annotations below for the details. :warning: Unable to include the failure links as the report was too long. See annotations below for the details.

Failing Jobs

Status	Name	Step	Failures	Logs	Raw logs	Build scan
✖	JVM Tests - JDK 17	`Build`	Failures	Logs	Raw logs	:construction:
✖	JVM Tests - JDK 17 Windows	`Build`	Failures	Logs	Raw logs	:construction:
✖	JVM Tests - JDK 21	`Build`	Failures	Logs	Raw logs	:construction:
✖	MicroProfile TCKs Tests	`Verify`	Failures	Logs	Raw logs	:construction:
✖	Native Tests - Data5	`Build`	Failures	Logs	Raw logs	:construction:

Full information is available in the Build summary check run.

:warning: Errors occurred while downloading the build reports. This report is incomplete.

Failures

:gear: JVM Tests - JDK 17 #

- Failing: integration-tests/kotlin integration-tests/narayana-lra

:package: integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin line 74 - History

:package: integration-tests/narayana-lra

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRA line 63 - History

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd line 44 - History

:gear: JVM Tests - JDK 17 Windows #

- Failing: integration-tests/kotlin

:package: integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin line 74 - History

:gear: JVM Tests - JDK 21 #

- Failing: integration-tests/kotlin integration-tests/narayana-lra

:package: integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinCreateMavenProjectIT.testProjectGenerationFromScratchForKotlin line 74 - History

:package: integration-tests/narayana-lra

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRA line 63 - History

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd line 44 - History

:gear: MicroProfile TCKs Tests #

- Failing: tcks/microprofile-lra

:package: tcks/microprofile-lra

✖ org.eclipse.microprofile.lra.tck.TckCancelOnTests.cancelFromRemoteCall line 160 - History

✖ org.eclipse.microprofile.lra.tck.TckCancelOnTests.cancelOn301 line 120 - History

✖ org.eclipse.microprofile.lra.tck.TckCancelOnTests.cancelOnFamily3xx line 100 - History

✖ org.eclipse.microprofile.lra.tck.TckCancelOnTests.cancelOnFamilyDefault4xx line 68 - History

✖ org.eclipse.microprofile.lra.tck.TckCancelOnTests.cancelOnFamilyDefault5xx line 84 - History

✖ org.eclipse.microprofile.lra.tck.TckCancelOnTests.notCancelOnFamily5xx line 140 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testAfterLRAEnlistmentDuringClosingPhase line 297 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testAsync1Support line 263 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testAsync2Support line 272 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testAsync3Support line 282 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testBasicContextPropagation line 101 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testContextAfterRemoteCalls line 258 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testForget line 156 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testForgetCalledForNestedParticipantsWhenParentIsClosed line 220 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testLeave line 135 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testParentContextAvailable line 190 - History

✖ org.eclipse.microprofile.lra.tck.TckContextTests.testStatus line 112 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.mandatoryEndWithLRA line 195 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.mandatoryEndWithLRAAtInterface line 346 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.mandatoryEndWithLRAAtSuperclass line 497 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.mandatoryWithLRA line 118 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.mandatoryWithLRAAtInterface line 269 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.mandatoryWithLRAAtSuperclass line 420 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithEndLRA line 231 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithEndLRAAtInterface line 382 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithEndLRAAtSuperclass line 533 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithInvalidLRA line 160 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithInvalidLRAAtInterface line 311 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithInvalidLRAAtSuperclass line 462 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithLRA line 154 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithLRAAtInterface line 305 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.neverWithLRAAtSuperclass line 456 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.notSupportedEndWithRA line 219 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.notSupportedEndWithRAAtInterface line 370 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.notSupportedEndWithRAAtSuperclass line 521 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.notSupportedWithRA line 142 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.notSupportedWithRAAtInterface line 293 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.notSupportedWithRAAtSuperclass line 444 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredEndWithLRA line 171 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredEndWithLRAAtInterface line 322 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredEndWithLRAAtSuperclass line 473 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredEndWithoutLRA line 177 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredEndWithoutLRAAtInterface line 328 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredEndWithoutLRAAtSuperclass line 479 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredWithLRA line 94 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredWithLRAAtInterface line 245 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredWithLRAAtSuperclass line 396 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredWithoutLRA line 100 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredWithoutLRAAtInterface line 251 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiredWithoutLRAAtSuperclass line 402 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresEndNewWithLRA line 183 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresEndNewWithLRAAtInterface line 334 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresEndNewWithLRAAtSuperclass line 485 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresEndNewWithoutLRA line 189 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresEndNewWithoutLRAAtInterface line 340 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresEndNewWithoutLRAAtSuperclass line 491 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresNewWithLRA line 106 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresNewWithLRAAtInterface line 257 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresNewWithLRAAtSuperclass line 408 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresNewWithoutLRA line 112 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresNewWithoutLRAAtInterface line 263 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.requiresNewWithoutLRAAtSuperclass line 414 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.supportsEndWithLRA line 207 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.supportsEndWithLRAAtInterface line 358 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.supportsEndWithLRAAtSuperclass line 509 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.supportsWithLRA line 130 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.supportsWithLRAAtInterface line 281 - History

✖ org.eclipse.microprofile.lra.tck.TckLRATypeTests.supportsWithLRAAtSuperclass line 432 - History

✖ org.eclipse.microprofile.lra.tck.TckParticipantTests.cancelLraDuringBusinessMethod line 189 - History

✖ org.eclipse.microprofile.lra.tck.TckParticipantTests.testNonJaxRsCompletionStageResponseAndParticipantStatus line 169 - History

✖ org.eclipse.microprofile.lra.tck.TckParticipantTests.testNonJaxRsCompletionStageVoid line 143 - History

✖ org.eclipse.microprofile.lra.tck.TckParticipantTests.validSignaturesChainTest line 115 - History

✖ org.eclipse.microprofile.lra.tck.TckParticipantTests.validWebApplicationExceptionReturnedTest line 81 - History

✖ org.eclipse.microprofile.lra.tck.TckRecoveryTests.testCancelWhenParticipantIsRestarted line 128 - History

✖ org.eclipse.microprofile.lra.tck.TckRecoveryTests.testCancelWhenParticipantIsUnavailable line 168 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.acceptCancelTest line 375 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.acceptCloseTest line 370 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.cancelLRA line 98 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.closeLRA line 118 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.compensateMultiLevelNestedActivity line 173 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.completeMultiLevelNestedActivity line 168 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.dependentLRA line 302 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.join line 211 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.joinLRAViaHeader line 183 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.joinWithOneResourceDifferentMethodTwiceWithCancel line 462 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.joinWithOneResourceDifferentMethodTwiceWithClose line 482 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.joinWithOneResourceSameMethodTwiceWithCancel line 452 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.joinWithOneResourceSameMethodTwiceWithClose line 472 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.joinWithTwoResourcesWithCancel line 501 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.joinWithTwoResourcesWithClose line 492 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.leaveLRA line 268 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.mixedMultiLevelNestedActivity line 178 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.nestedActivity line 128 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.noLRATest line 422 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.testAfterLRAListener line 250 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.testAfterLRAParticipant line 229 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.timeLimit line 322 - History

✖ org.eclipse.microprofile.lra.tck.TckTests.timeLimitWithPreConditionFailed line 361 - History

✖ org.eclipse.microprofile.lra.tck.TckUnknownStatusTests.compensate_retry line 65 - History

✖ org.eclipse.microprofile.lra.tck.TckUnknownStatusTests.complete_retry line 79 - History

✖ org.eclipse.microprofile.lra.tck.TckUnknownTests.compensate_immediate line 74 - History

✖ org.eclipse.microprofile.lra.tck.TckUnknownTests.compensate_retry line 86 - History

✖ org.eclipse.microprofile.lra.tck.TckUnknownTests.complete_immediate line 99 - History

✖ org.eclipse.microprofile.lra.tck.TckUnknownTests.complete_retry line 111 - History

:gear: Native Tests - Data5 #

- Failing: integration-tests/jpa-postgresql-withxml

:package: integration-tests/jpa-postgresql-withxml

✖ io.quarkus.it.jpa.postgresql.ImageMetricsITCase.verifyImageMetrics line 15 - History

Feb 27 '24 15:02 quarkus-bot[bot]

I think it's a small price to pay to ask those to install GraalVM locally and run the native-with-agent profile with their unit tests.

I agree with @zakkak . I wish we did not have to install GraalVM on our build machines. today we do not. we are building the native image using the build image. and that saves a lot of pain, considering that many quarkus applications are using different versions of quarkus, hence different versions of graalvm/mandrel. at this time we target native first for all of our applications that do not use an excessive amount of memory, and/or do not the absolute best performances. that means that almost all of our quarkus applications target mandrel today. having this requirement would mean we have to install every single version of mandrel, and make sure we are using the right one for a given quarkus version. feasible, but I can see the eyes of the soft factory lead when I will be telling him that ;-) for now, I can provide some feedback on a special host I am using for my native tests, which has versions of graalvm installed, but going forward if this requirement stays, the feature is probably going to be used lightly.

Feb 27 '24 16:02 vsevel

I agree with @zakkak . I wish we did not have to install GraalVM on our build machines. today we do not. we are building the native image using the build image. and that saves a lot of pain, considering that many quarkus applications are using different versions of quarkus, hence different versions of graalvm/mandrel.

Fair enough. I'll explore this option to see how JVM mode unit tests can run with the GraalVM installation within the builder image.

Feb 28 '24 09:02 galderz

If I understand it right I would say we should fix or at least make it easier to match the right graalvm/mandrel with quarkus than keep doing things that is only possible with docker.

Remember that the native agent will potentially record different things dependent on which OS it is running on.

Thus I agree it would be interesting to be able to run the tests via a JVM inside docker but that feels like secondary to this PR's main usecase of optionally allow running native agent?

Feb 28 '24 11:02 maxandersen

We had a long discussion about this on Zulip.

Feb 28 '24 11:02 geoand

but that feels like secondary to this PR's main usecase of optionally allow running native agent?

I agree this is secondary. this PR brings a lot of value, even without the capability to not rely on a local installation of graalvm. it is not part of the mvp. for us it is the difference between getting it to run every time for every app, and from time to time when we have an issue with the app, or the very first time we target native. I doubt we will be able to convince the soft factory team to follow up with installing the various versions of graalvm we need (today we do everything in docker, including maven), but that may be different in other organizations. another option for us is to convince developers to install graalvm on their dev machine. but they will do so only if they really need to (even if I write a procedure). the fact that we are on windows, adds the extra requirement that we have to install the vs tools. so all I am saying is that requiring a local graalvm install will limit the number of times we are using this feature (whether or not we look at the build result). my initial thought was that I would add this profile to our default quarkus build pipeline. I won't be able to do that. but like I said, there is already a lot of value, even without docker support.

Feb 28 '24 11:02 vsevel

:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit e1f518a61dc59ef59fdf7a4248bb7aee558883d7.

:warning: Unable to include the stracktraces as the report was too long. See annotations below for the details. :warning: Unable to include the failure links as the report was too long. See annotations below for the details.

Failing Jobs

Status	Name	Step	Failures	Logs	Raw logs	Build scan
✖	JVM Tests - JDK 17	`Build`	Failures	Logs	Raw logs	:construction:
✖	JVM Tests - JDK 21	`Build`	Failures	Logs	Raw logs	:construction:
✖	MicroProfile TCKs Tests	`Verify`	Failures	Logs	Raw logs	:construction:

Full information is available in the Build summary check run.

Failures

:gear: JVM Tests - JDK 17 #

- Failing: integration-tests/narayana-lra

:package: integration-tests/narayana-lra

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRA line 63 - History

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd line 44 - History

:gear: JVM Tests - JDK 21 #

- Failing: integration-tests/narayana-lra

:package: integration-tests/narayana-lra

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRA line 63 - History

✖ org.acme.quickstart.lra.LRAParticipantTest.testLRAStartEnd line 44 - History

:gear: MicroProfile TCKs Tests #

- Failing: tcks/microprofile-lra