elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

Evaluate Early Access builds of JDK 22.0.2

Open ChrisHegarty opened this issue 1 year ago • 6 comments

This issue is intended to help track the evaluation of JDK 22.0.2 Early Access builds, in order to:

  • [x] Confirm that the G1 Hotspot JVM bug has been resolved, see #106987
  • [x] Ensure that Elasticsearch is stable on this yet-to-be-released JDK 22 update.
  • [x] Evaluate the workaround by setting, #108571 -XX:+UnlockDiagnosticVMOptions -XX:G1NumCollectionsKeepPinned=10000000

The motivation for this work is to enable Elasticsearch to bundle the JDK 22.0.2 when it is released ( expected 2024-07-16, see https://www.java.com/releases/ )

relates #106987

ChrisHegarty avatar Apr 27 '24 14:04 ChrisHegarty

Pinging @elastic/es-core-infra (Team:Core/Infra)

elasticsearchmachine avatar Apr 27 '24 14:04 elasticsearchmachine

The G1 issue is reproducible on Linux x64 with the nyc_taxis benchmark, see https://github.com/elastic/elasticsearch-benchmarks/issues/1989.

We're currently running this benchmark with a locally built JDK 22.0.2 that includes the JDK bug fix https://bugs.openjdk.org/browse/JDK-8329528. The benchmark is run over and over in a loop, just evaluating that Elasticsearch does not crash. So far, it passes successfully. This issue will be updated with more specifics when they are available.

ChrisHegarty avatar Apr 27 '24 14:04 ChrisHegarty

Pinging @elastic/es-perf (Team:Performance)

elasticsearchmachine avatar Apr 27 '24 16:04 elasticsearchmachine

We've run about 40 iterations so far of the benchmark, even lowering the heap further, however the issue has not reoccurred.

Update 30/4 - 67 iterations and still going

gareth-ellis avatar Apr 29 '24 12:04 gareth-ellis

The next step is to evaluate JDK 22.0.1 with the "JVM options" workaround. That is with:

-XX:+UnlockDiagnosticVMOptions -XX:G1NumCollectionsKeepPinned=10000000

ChrisHegarty avatar May 10 '24 13:05 ChrisHegarty

We ran all weekend with the JVM args and JDK 22.0.1 - completed 35 iterations without any issues. I then removed the argument and saw the JVM crash - so I'm happy that adding the two JVM args seem to stop the issue occurring

gareth-ellis avatar May 13 '24 07:05 gareth-ellis