openj9 icon indicating copy to clipboard operation
openj9 copied to clipboard

Published 20 hours ago • verified publisher icon eclipse-openj9

jdk_util_0 java/util/stream/test/org/openjdk/tests/java/util/stream/MapOpTest.java Segmentation error vmState=0x00000000

Open JasonFengJ9 opened this issue 6 months ago • 6 comments

Failure link

From internal Test_openjdk24_j9_sanity.openjdk_ppc64le_linux_testList_0 (rtj-ubu24le-rtp-test-932vb-1)

openjdk version "24.0.1-beta" 2025-04-15
IBM Semeru Runtime Open Edition 24.0.1+9-202505241603 (build 24.0.1-beta+9-202505241603)
Eclipse OpenJ9 VM 24.0.1+9-202505241603 (build master-7bed029788, JRE 24 Linux ppc64le-64-Bit Compressed References 20250524_67 (JIT enabled, AOT enabled)
OpenJ9   - 7bed029788
OMR      - 988ae21a0
JCL      - 66c0fb7ed based on jdk-24.0.1+9)

Rerun in Grinder - Change TARGET to run only the failed test targets

Optional info

Failure output (captured from console output)

[2025-05-24T18:41:47.153Z] variation: -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache Mode150
[2025-05-24T18:41:47.153Z] JVM_OPTIONS:  -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:+UseCompressedOops -Xverbosegclog 

[2025-05-24T18:51:26.790Z] TEST: java/util/stream/test/org/openjdk/tests/java/util/stream/MapOpTest.java

[2025-05-24T18:51:26.799Z] STDERR:
[2025-05-24T18:51:26.799Z] Unhandled exception
[2025-05-24T18:51:26.799Z] Type=Segmentation error vmState=0x00000000
[2025-05-24T18:51:26.799Z] Unhandled exception
[2025-05-24T18:51:26.799Z] Type=Segmentation error vmState=0x00000000
[2025-05-24T18:51:26.799Z] J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
[2025-05-24T18:51:26.799Z] Handler1=00007D4167353BB0 Handler2=00007D4167278220
[2025-05-24T18:51:26.799Z] J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
[2025-05-24T18:51:26.799Z] Handler1=00007D4167353BB0 Handler2=00007D4167278220

[2025-05-24T18:51:26.801Z] 
[2025-05-24T18:51:26.801Z] Compiled_method=java/util/stream/ForEachOps$ForEachTask.compute()V
[2025-05-24T18:51:26.801Z] Target=2_90_20250524_67 (Linux 6.8.0-59-generic)
[2025-05-24T18:51:26.801Z] CPU=ppc64le (32 logical CPUs) (0x1f7d90000 RAM)
[2025-05-24T18:51:26.801Z] ----------- Stack Backtrace -----------
[2025-05-24T18:51:26.801Z] 
[2025-05-24T18:51:26.801Z] Compiled_method=java/util/stream/ForEachOps$ForEachTask.compute()V
[2025-05-24T18:51:26.801Z] Target=2_90_20250524_67 (Linux 6.8.0-59-generic)
[2025-05-24T18:51:26.801Z] CPU=ppc64le (32 logical CPUs) (0x1f7d90000 RAM)
[2025-05-24T18:51:26.801Z] ----------- Stack Backtrace -----------
[2025-05-24T18:51:26.801Z]  (0x00007D41445AA71C [<unknown>+0x0])
[2025-05-24T18:51:26.801Z] runJavaThread+0x250 (0x00007D4167328B50 [libj9vm29.so+0x18b50])
[2025-05-24T18:51:26.801Z] javaProtectedThreadProc+0xc8 (0x00007D41673CF468 [libj9vm29.so+0xbf468])
[2025-05-24T18:51:26.801Z] omrsig_protect+0x3e4 (0x00007D41672796D4 [libj9prt29.so+0x396d4])
[2025-05-24T18:51:26.801Z] javaThreadProc+0x60 (0x00007D41673CA900 [libj9vm29.so+0xba900])
[2025-05-24T18:51:26.801Z] thread_wrapper+0x190 (0x00007D416720CC00 [libj9thr29.so+0xcc00])
[2025-05-24T18:51:26.801Z]  (0x00007D4167D12A9C [libc.so.6+0xb2a9c])
[2025-05-24T18:51:26.801Z] ---------------------------------------
[2025-05-24T18:51:26.801Z]  (0x00007D41445AA71C [<unknown>+0x0])
[2025-05-24T18:51:26.801Z] runJavaThread+0x250 (0x00007D4167328B50 [libj9vm29.so+0x18b50])
[2025-05-24T18:51:26.801Z] javaProtectedThreadProc+0xc8 (0x00007D41673CF468 [libj9vm29.so+0xbf468])
[2025-05-24T18:51:26.801Z] omrsig_protect+0x3e4 (0x00007D41672796D4 [libj9prt29.so+0x396d4])
[2025-05-24T18:51:26.801Z] javaThreadProc+0x60 (0x00007D41673CA900 [libj9vm29.so+0xba900])
[2025-05-24T18:51:26.801Z] thread_wrapper+0x190 (0x00007D416720CC00 [libj9thr29.so+0xcc00])
[2025-05-24T18:51:26.801Z]  (0x00007D4167D12A9C [libc.so.6+0xb2a9c])
[2025-05-24T18:51:26.801Z] ---------------------------------------

50x internal Grinder - the failure wasn't reproduced

JasonFengJ9 avatar May 27 '25 13:05 JasonFengJ9

garbage in O-slots:

Checking THREAD STACKS...
  <gc check (1): from debugger: THREAD STACKS: slot 505a00(7d4143a5b7c8) -> 6ace0019: corrupt data exception>
  <gc check (2): from debugger: THREAD STACKS: slot 505a00(7d4143a5b7c0) -> 6ace0019: corrupt data exception>
  <gc check (3): from debugger: THREAD STACKS: slot 505a00(7d4143a5b7b0) -> 13001fa742003d: corrupt data exception>
  <gc check (4): from debugger: THREAD STACKS: slot 505a00(7d4143a5b798) -> 6acc0019: corrupt data exception>
  <gc check (5): from debugger: THREAD STACKS: slot 505a00(7d4143a5b778) -> ffffffffffffffff: corrupt data exception>
  <gc check (6): from debugger: THREAD STACKS: slot 524400(7d414366b7c8) -> ec86002b: corrupt data exception>
  <gc check (7): from debugger: THREAD STACKS: slot 524400(7d414366b7c0) -> ec86002b: corrupt data exception>
  <gc check (8): from debugger: THREAD STACKS: slot 524400(7d414366b7b0) -> a001f20260005: corrupt data exception>
  <gc check (9): from debugger: THREAD STACKS: slot 524400(7d414366b798) -> ec84002b: corrupt data exception>
  <gc check (10): from debugger: THREAD STACKS: slot 524400(7d414366b778) -> ffffffffffffffff: corrupt data exception>

<524400> JIT frame: bp = 0x00000000005D4D48, pc = 0x00007D41445AA71C, unwindSP = 0x00000000005D4930, cp = 0x00000000005EDA70, arg0EA = 0x00000000005D4D50, jitInfo = 0x00007D410CE7C0B8
<524400> 	Method: java/util/stream/ForEachOps$ForEachTask.compute()V !j9method 0x00000000005EDC48
<524400> 	Bytecode index = 49, inlineDepth = 0, PC offset = 0x00000000000037A0
<524400> 	stackMap=0x00007D410CE7DB6E, slots=I16(0x0001) parmBaseOffset=I16(0x0008), parmSlots=U16(0x0001), localBaseOffset=I16(0xFE38)
<524400> 	Described JIT args starting at 0x00000000005D4D50 for U16(0x0001) slots
<524400> 		O-Slot: : a0[0x00000000005D4D50] = 0x00000000FFC5BCB0
<524400> 	Described JIT temps starting at 0x00000000005D4B80 for IDATA(0x0000000000000039) slots
<524400> 		I-Slot: : t56[0x00000000005D4B80] = 0x00000000FFC454D0
<524400> 		I-Slot: : t55[0x00000000005D4B88] = 0x0000000000600200
<524400> 		I-Slot: : t54[0x00000000005D4B90] = 0x00007D4165CCDCE8
<524400> 		I-Slot: : t53[0x00000000005D4B98] = 0x00007D414409F708
<524400> 		I-Slot: : t52[0x00000000005D4BA0] = 0x0000000000600000
<524400> 		I-Slot: : t51[0x00000000005D4BA8] = 0x0000000000600000
<524400> 		I-Slot: : t50[0x00000000005D4BB0] = 0x000000009FC00000
<524400> 		I-Slot: : t49[0x00000000005D4BB8] = 0x00000000FFC454D0
<524400> 		I-Slot: : t48[0x00000000005D4BC0] = 0x000000009FC4EC98
<524400> 		I-Slot: : t47[0x00000000005D4BC8] = 0x00007D4144476730
<524400> 		I-Slot: : t46[0x00000000005D4BD0] = 0x00000000FFC45448
<524400> 		I-Slot: : t45[0x00000000005D4BD8] = 0x0000000000538400
<524400> 		I-Slot: : t44[0x00000000005D4BE0] = 0x0000000000000001
<524400> 		I-Slot: : t43[0x00000000005D4BE8] = 0x00000000FFC5BCB0
<524400> 		I-Slot: : t42[0x00000000005D4BF0] = 0x00000000FFC227B8
<524400> 		I-Slot: : t41[0x00000000005D4BF8] = 0x00000000FFC227D0
<524400> 		I-Slot: : t40[0x00000000005D4C00] = 0x00000000FFC401E8
<524400> 		I-Slot: : t39[0x00000000005D4C08] = 0x00000000FFC227B8
<524400> 		I-Slot: : t38[0x00000000005D4C10] = 0x00000000FFC401E8
<524400> 		I-Slot: : t37[0x00000000005D4C18] = 0x00000000FFC227B8
<524400> 		I-Slot: : t36[0x00000000005D4C20] = 0x000000009FC3AD68
<524400> 		I-Slot: : t35[0x00000000005D4C28] = 0x00000000FFC40240
<524400> 		I-Slot: : t34[0x00000000005D4C30] = 0x00000000FFC454D0
<524400> 		I-Slot: : t33[0x00000000005D4C38] = 0x00000000000EBA30
<524400> 		I-Slot: : t32[0x00000000005D4C40] = 0x000000009FE54920
<524400> 		I-Slot: : t31[0x00000000005D4C48] = 0x000000009FE54920
<524400> 		I-Slot: : t30[0x00000000005D4C50] = 0x0000000000000001
<524400> 		I-Slot: : t29[0x00000000005D4C58] = 0x0000000000000000
<524400> 		I-Slot: : t28[0x00000000005D4C60] = 0x0000000000000001
<524400> 		I-Slot: : t27[0x00000000005D4C68] = 0x000000009FE55378
<524400> 		I-Slot: : t26[0x00000000005D4C70] = 0x0000000000000001
<524400> 		I-Slot: : t25[0x00000000005D4C78] = 0x0000000000229900
<524400> 		I-Slot: : t24[0x00000000005D4C80] = 0x000000009FC3AD68
<524400> 		I-Slot: : t23[0x00000000005D4C88] = 0x0000000000000000
<524400> 		I-Slot: : t22[0x00000000005D4C90] = 0x0000000000000001
<524400> 		I-Slot: : t21[0x00000000005D4C98] = 0x000000009FE549C8
<524400> 		I-Slot: : t20[0x00000000005D4CA0] = 0xFFFFFFFFFFFFFFFF
<524400> 		I-Slot: : t19[0x00000000005D4CA8] = 0x0000000000000000
<524400> 		I-Slot: : t18[0x00000000005D4CB0] = 0x0000000000000000
<524400> 		I-Slot: : t17[0x00000000005D4CB8] = 0x0000000000000000
<524400> 		I-Slot: : t16[0x00000000005D4CC0] = 0x00000000EC84002B
<524400> 		I-Slot: : t15[0x00000000005D4CC8] = 0x00000000EC85002B
<524400> 		I-Slot: : t14[0x00000000005D4CD0] = 0x000000009FE549C8
<524400> 		I-Slot: : t13[0x00000000005D4CD8] = 0x000A001F20260005
<524400> 		I-Slot: : t12[0x00000000005D4CE0] = 0x000000009FD76D88
<524400> 		I-Slot: : t11[0x00000000005D4CE8] = 0x00000000EC86002B
<524400> 		I-Slot: : t10[0x00000000005D4CF0] = 0x00000000EC86002B
<524400> 		I-Slot: : t9[0x00000000005D4CF8] = 0x000000009FC3AD68
<524400> 		I-Slot: : t8[0x00000000005D4D00] = 0x000000009FE54920
<524400> 		I-Slot: : t7[0x00000000005D4D08] = 0x000000000000004C
<524400> 		I-Slot: : t6[0x00000000005D4D10] = 0x0000000000003D65
<524400> 		O-Slot: : t5[0x00000000005D4D18] = 0x000000009FC44428
<524400> 		I-Slot: : t4[0x00000000005D4D20] = 0x00000000FFC5BCB0
<524400> 		I-Slot: : t3[0x00000000005D4D28] = 0x000000009FE55378
<524400> 		I-Slot: : t2[0x00000000005D4D30] = 0x000000000000003F
<524400> 		I-Slot: : t1[0x00000000005D4D38] = 0x00000000FFC45508
<524400> 		I-Slot: : t0[0x00000000005D4D40] = 0x00000000FFC337C8
<524400> 	JIT-RegisterMap = UDATA(0x0000000000004DB4)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B7E8] = UDATA(0x000000009FE466A0) (jit_r31)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B7E0] = UDATA(0x000000000000004C) (jit_r30)
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B7D8] = 0x000000009FE54920 (jit_r29)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B7D0] = UDATA(0x000000009FC3AD68) (jit_r28)
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B7C8] = 0x00000000EC86002B (jit_r27) <-------------
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B7C0] = 0x00000000EC86002B (jit_r26) <-------------
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B7B8] = UDATA(0x000000009FD76D88) (jit_r25)
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B7B0] = 0x000A001F20260005 (jit_r24) <--------------
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B7A8] = 0x000000009FE549C8 (jit_r23)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B7A0] = UDATA(0x00000000EC85002B) (jit_r22)
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B798] = 0x00000000EC84002B (jit_r21) <--------------
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B790] = 0x0000000000000000 (jit_r20)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B788] = UDATA(0x0000000000000000) (jit_r19)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B780] = UDATA(0x0000000000000000) (jit_r18)
<524400> 		JIT-RegisterMap-O-Slot[0x00007D414366B778] = 0xFFFFFFFFFFFFFFFF (jit_r17) <-----------------
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B770] = UDATA(0x00007D41427E0038) (jit_r16)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B768] = UDATA(0x0000000000000000) (jit_r15)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B760] = UDATA(0x0000000000000000) (jit_r14)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B758] = UDATA(0x00007D41436768E0) (jit_r13)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B750] = UDATA(0x0000000000238D00) (jit_r12)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B748] = UDATA(0x00007D4144476730) (jit_r11)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B740] = UDATA(0x000000000063CE00) (jit_r10)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B738] = UDATA(0x0000000000239D08) (jit_r9)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B730] = UDATA(0x0000000000600000) (jit_r8)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B728] = UDATA(0x0000000000600000) (jit_r7)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B720] = UDATA(0x000000009FC00000) (jit_r6)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B718] = UDATA(0x00007D4144476730) (jit_r5)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B710] = UDATA(0x00007D4144476748) (jit_r4)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B708] = UDATA(0x000000000063CE00) (jit_r3)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B700] = UDATA(0x00007D4165FEE700) (jit_r2)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B6F8] = UDATA(0x0000000000000000) (jit_r1)
<524400> 		JIT-RegisterMap-I-Slot[0x00007D414366B6F0] = UDATA(0x00000000001DB300) (jit_r0)
<524400> 	JIT-Frame-RegisterMap[0x00007D414366B770] = UDATA(0x00007D41427E0038) (jit_r16)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D4968] = UDATA(0x0000000000000000) (jit_r17)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D4970] = UDATA(0x0000000000000000) (jit_r18)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D4978] = UDATA(0x0000000000000000) (jit_r19)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D4980] = UDATA(0x0000000000000000) (jit_r20)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D4988] = UDATA(0x00000000FFC5BCB0) (jit_r21)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D4990] = UDATA(0x000000009FE549C8) (jit_r22)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D4998] = UDATA(0x000000009FE549C8) (jit_r23)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49A0] = UDATA(0x000A001F20260005) (jit_r24)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49A8] = UDATA(0x000000009FD76D88) (jit_r25)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49B0] = UDATA(0x00000000EC86002B) (jit_r26)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49B8] = UDATA(0x00000000EC86002B) (jit_r27)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49C0] = UDATA(0x000000009FC3AD68) (jit_r28)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49C8] = UDATA(0x000000009FE54920) (jit_r29)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49D0] = UDATA(0x00000000005ED700) (jit_r30)
<524400> 	JIT-Frame-RegisterMap[0x00000000005D49D8] = UDATA(0x0000000000000000) (jit_r31)

dmitripivkine avatar May 27 '25 14:05 dmitripivkine

FYI @hzongaro

tajila avatar May 27 '25 15:05 tajila

@dylanjtuttle, may I ask you to look at this problem?

hzongaro avatar Jun 03 '25 21:06 hzongaro

Update from Dylan:

I spent some time trying to learn something about the failure using the core file and jitdump provided in the issue, but so far I haven’t been able to figure out much.

I switched my focus over to trying to reproduce the failure when Henry mentioned to me that Tobi suspects that the issue might not be JDK 24 specific. To this point, I have been unable to reproduce the failure, either in JDK 24 or out. The issue mentions that the failure did not appear in a 50 iteration grinder run either, so at the very least it does seem somewhat uncommon.

vij-singh avatar Jun 12 '25 15:06 vij-singh

@JasonFengJ9 Has this been reproduced after the original failure?

vij-singh avatar Jun 12 '25 15:06 vij-singh

@vij-singh The original failure wasn't reproduced in the grinder, iirc, it wasn't seen in recent builds.

Moving it out milestone plan.

JasonFengJ9 avatar Jun 12 '25 15:06 JasonFengJ9

https://openj9-jenkins.osuosl.org/job/Test_openjdk24_j9_sanity.openjdk_ppc64le_linux_Nightly_testList_2/126/consoleFull

22:19:05  Running test jdk_util_1 ...
22:19:05  ===============================================
22:19:05  jdk_util_1 Start Time: Fri Jul 25 02:19:05 2025 Epoch Time (ms): 1753409945581
22:19:05  variation: -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache Mode650
22:19:05  JVM_OPTIONS:  -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-UseCompressedOops -Xverbosegclog 

22:25:34  TEST: java/util/stream/test/org/openjdk/tests/java/util/stream/MapOpTest.java

22:25:34  Unhandled exception
22:25:34  Type=Segmentation error vmState=0x00000000

22:25:34  Compiled_method=java/util/stream/ForEachOps$ForEachTask.compute()V
22:25:34  Target=2_90_20250725_130 (Linux 5.14.0-542.el9.ppc64le)
22:25:34  CPU=ppc64le (8 logical CPUs) (0x3975f0000 RAM)
22:25:34  ----------- Stack Backtrace -----------
22:25:34   (0x00007FFF37FCFAA8 [<unknown>+0x0])
22:25:34  runJavaThread+0x250 (0x00007FFFBA018BF0 [libj9vm29.so+0x18bf0])
22:25:34  javaProtectedThreadProc+0xc8 (0x00007FFFBA0C01C8 [libj9vm29.so+0xc01c8])
22:25:34  omrsig_protect+0x3e4 (0x00007FFFBA3696D4 [libj9prt29.so+0x396d4])
22:25:34  javaThreadProc+0x60 (0x00007FFFBA0BB640 [libj9vm29.so+0xbb640])
22:25:34  thread_wrapper+0x190 (0x00007FFFBA75CC00 [libj9thr29.so+0xcc00])
22:25:34  start_thread+0x170 (0x00007FFFBAAA2FB0 [libc.so.6+0xa2fb0])
22:25:34  clone+0xa0 (0x00007FFFBAB4C7C8 [libc.so.6+0x14c7c8])
22:25:34  ---------------------------------------

It might be related to

  • https://github.com/eclipse-openj9/openj9/issues/21824

JasonFengJ9 avatar Jul 25 '25 11:07 JasonFengJ9

This probably isn’t related to #21824, as the remaining problem there seems to be specific to aarch64.

hzongaro avatar Jul 30 '25 20:07 hzongaro

I haven't had any luck reproducing this failure in grinder runs. I'm going to remove this from the 0.56 release for now. If it's seen again, we can resume the investigation.

hzongaro avatar Aug 18 '25 15:08 hzongaro