graal icon indicating copy to clipboard operation
graal copied to clipboard

Significant execution time degradation with agent used during tests

Open kgonia opened this issue 1 year ago • 8 comments

Describe the issue When running Spring tests with the GraalVM agent, there is significant execution time degradation compared to running without the agent. With the agent enabled, test execution takes over 1 hour versus 4 minutes without it for a project with ~120 MockMvc integration tests.

Originally I described issue in gradle native plugin https://github.com/graalvm/native-build-tools/issues/655 but was suggested to move it here.

Steps to reproduce the issue

  1. The issue can be partially demonstrated with spring-petclinic, though with smaller impact:
    • Without agent: ~1m 28s
    • With standard agent: ~2m 16s
    • With conditional agent: ~3m 43s

However, my case shows much more severe degradation. I should note that a few beans were mocked using @MockBean (switching to @MockitoBean didn't change much), which might lead to context refresh. Even considering this behavior, I'm wondering why it would slow execution so dramatically. Additionally, pet-clinic also uses @MockitoBean but doesn't show comparable degradation.

Environment: OS: Windows GraalVM Version: GraalVM 22.3.1 Java 19 C Java Version: 19.0.2+7-jvmci-22.3-b12

OS: WSL GraalVM Version: CE 23.0.1+11.1 Java Version: 23.0.1

kgonia avatar Dec 19 '24 12:12 kgonia

Hello, any news ? I have that issue too, by using the agent along with the integration-tests, it takes 2 times more with 25.0.1 compared to 24.0.1. ( spring boot 3.5.6, with kafka test, h2, mongo, test-containers).

lduboeuf avatar Oct 30 '25 20:10 lduboeuf

Thank you for reporting. It is a known issue that the current agent is taking too long in some cases as it is based on debugging breakpoints. In some benchmarks the performance can drop even by 10x.

We are working on a new tracing which does not use breakpoints, and exactly according to the Native Image semantics. This will fix the problem, but it will arrive in the following releases (25.x). It will likely not be backported to 25.0.

vjovanov avatar Nov 29 '25 06:11 vjovanov

@vjovanov is there an issue we can follow for this ongoing work? I’m seeing similar degradation (~2.5x slower) after upgrading from 24 to 25.0.

rotilho avatar Dec 01 '25 08:12 rotilho

We can use this one: https://github.com/oracle/graal/issues/9812

vjovanov avatar Dec 01 '25 19:12 vjovanov

That proposed change would be awesome, I can see how massive the improvement would be… but it looks like something is off with the latest release.

On my side I’m seeing a big slowdown when running with -Pagent after moving to GraalVM 25. Unfortunately I also upgraded Kotlin and Spring at the same time, so I can’t say with 100% certainty the regression is solely due to the GraalVM change, but the difference is pretty dramatic when compared with no -Pagent:

  • GraalVM 24 (with -Pagent): ~11 minutes (source: Action)
  • GraalVM 25 (with -Pagent): ~37 minutes (source: Action)
  • GraalVM 25 (without -Pagent): ~4 minutes (source: Action)

I noticed this repo builds against popular frameworks. Have you seen any similar degradation when running with -Pagent on GraalVM 25 compared to 24?

rotilho avatar Dec 02 '25 08:12 rotilho

Not yet, this is the first report of this kind. A clear reproducer would help us identify what is the cause. Could you share it here for completeness?

vjovanov avatar Dec 02 '25 17:12 vjovanov

Sure thing! I ran a few times to compare them: https://github.com/rotilho/spring-petclinic/actions/runs/19873594878

On above forked spring-petclinic project, GraalVM 25 is consistently about 1.5x-2x slower than 21 and 24.

On a larger codebase like https://github.com/attocash/node, the slowdown seems even more extreme, closer to 3x in my tests.

rotilho avatar Dec 02 '25 21:12 rotilho

I found another potential issue (https://github.com/oracle/graal/issues/12650), so I prepared another example with a larger repository. The results:

21: 9m12s 24: 8m52s 25: 35m56s

Source: https://github.com/rotilho/node-reproducer/actions/runs/19893636797

rotilho avatar Dec 03 '25 13:12 rotilho

I noticed on other project that issue might be with gradle and graalvm interoperability. Even without agent it's terribly slow when run using gradle.

When I run project on graalvm with intellij it's start much faster. App is loading heavy python libs using graalpy.

2025-12-12 13:50:51 - Root WebApplicationContext: initialization completed in 1100 ms
2025-12-12 13:53:39 - Started ImageoApplication in 170.094 seconds (process running for 171.129)

with gradle :bootRun (also with --no-deamon)

2025-12-12 13:45:33 - Root WebApplicationContext: initialization completed in 8485 ms

Affter 16 minutes I stoped waiting for app to start...

I created also maven config for my project

2025-12-12 14:56:22 - Root WebApplicationContext: initialization completed in 596 ms
2025-12-12 14:56:38 - Started ImageoApplication in 16.631 seconds (process running for 16.918)

kgonia avatar Dec 12 '25 13:12 kgonia

I tried to reproduce on spring pet clinic using agent. Diffrence is noticable but not as severe.

gradle: ./gradlew -Pagent test

BUILD SUCCESSFUL in 3m 47s`

maven ./mvnw -Pnative -Dagent=true test

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:09 min

kgonia avatar Dec 12 '25 14:12 kgonia