Fix null restricted array related issues for value types

[a7ehuo]

(1) Add utility methods for null restricted arrays (2) Fix null restricted array related issues

• Add recognized method jdk/internal/value/ValueClass.newArrayInstance
• Add VP fixed class constraint if ValueClass.newArrayInstance creates null restricted array
• Rename isArrayCompTypePrimitiveValueType to isArrayNullRestricted to reflect the updated logic
• Update isArrayNullRestricted on how to determine whether or not an array is a null restricted array
• Disable transformation based on type hint which is no longer sufficient enough to decide whether or not the array is null restricted or not
• If flattenable arrays are enabled, do not create fixed class constraint for parm array class based on signature since both nullable and null restricted arrays share the same signature

Depends on

• [x] https://github.com/eclipse-openj9/openj9/pull/20132
• [x] https://github.com/eclipse-openj9/openj9/pull/20139
• [x] https://github.com/eclipse-openj9/openj9/pull/19995
• [x] https://github.com/eclipse-openj9/openj9/pull/20086
• [ ] https://github.com/eclipse/omr/pull/7452

Related: #19913, #19914, #19708

[a7ehuo]

@hzongaro May I ask you to review this change? Thank you very much!

This PR is created as a draft and marked as WIP for now because of all the dependencies. Regarding to the implementation itself, it is ready for review.

Since this PR depends on https://github.com/eclipse/omr/pull/7452, it should be review along with https://github.com/eclipse/omr/pull/7452.

[a7ehuo]

@hzongaro All comments have been addressed in the latest commits, except for the following three that will be addressed in the future PRs due to the complexity. Ready for another review. Thank you!

(1) https://github.com/eclipse-openj9/openj9/pull/20112#discussion_r1750674700 (2) https://github.com/eclipse-openj9/openj9/pull/20112#discussion_r1750678052 (3) https://github.com/eclipse-openj9/openj9/pull/20112#discussion_r1750726230

Please note that I have rebased this PR to the latest master branch and this PR requires coordinated merge with https://github.com/eclipse/omr/pull/7452.

[a7ehuo]

@hzongaro All comments are addressed. Ready for another review. Thank you!

I rebased the code in order to resolve the merge conflict in CommunicationStream.hpp

[a7ehuo]

@hzongaro All comments are addressed. Ready for another review. Thank you!

[hzongaro]

Jenkins test sanity.functional,extended.functional xlinuxval,plinuxval,zlinuxval,alinuxval jdknext depends eclipse/omr#7452

[hzongaro]

Jenkins test sanity.functional,sanity.openjdk xlinuxvalst,plinuxvalst,zlinuxvalst,alinuxvalst jdknext depends eclipse/omr#7452

[hzongaro]

Jenkins test sanity.functional,sanity.openjdk all jdk8,jdk11,jdk17,jdk21 depends eclipse/omr#7452

[a7ehuo]

Test_openjdk11_j9_sanity.openjdk_aarch64_linux_Personal: Known issue https://github.com/eclipse-openj9/openj9/issues/13756

[2024-10-15T20:03:29.491Z] java.lang.Exception
[2024-10-15T20:03:29.491Z] 	at ReplayCacheTestProc.main0(ReplayCacheTestProc.java:279)
[2024-10-15T20:03:29.491Z] 	at ReplayCacheTestProc.main(ReplayCacheTestProc.java:326)
[2024-10-15T20:03:29.491Z] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2024-10-15T20:03:29.491Z] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[2024-10-15T20:03:29.491Z] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[2024-10-15T20:03:29.491Z] 	at java.base/java.lang.reflect.Method.invoke(Method.java:572)
[2024-10-15T20:03:29.491Z] 	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
[2024-10-15T20:03:29.491Z] 	at java.base/java.lang.Thread.run(Thread.java:839)
[2024-10-15T20:03:29.491Z] java.lang.Exception
[2024-10-15T20:03:29.491Z] 	at ReplayCacheTestProc.main0(ReplayCacheTestProc.java:279)
[2024-10-15T20:03:29.491Z] 	at ReplayCacheTestProc.main(ReplayCacheTestProc.java:326)
...
[2024-10-15T20:04:55.398Z] jdk_security4_0_FAILED

Test_openjdk11_j9_sanity.openjdk_x86-64_windows_Personal: Known issue https://github.com/eclipse-openj9/openj9/issues/17743

[2024-10-16T02:47:35.873Z] java.lang.RuntimeException: PortUnreachableException not thrown.
[2024-10-16T02:47:35.873Z] 	at Unreachable.main(Unreachable.java:102)
...
[2024-10-16T02:52:31.349Z] jdk_security4_1_FAILED

Test_openjdk17_j9_sanity.openjdk_ppc64_aix_Personal: Known issue: https://github.com/eclipse-openj9/openj9/issues/19962

[2024-10-15T22:10:03.705Z] ACTION: main -- Error. Agent communication error: java.io.EOFException; check console log for any additional details
[2024-10-15T22:10:03.705Z] REASON: Assumed action based on file name: run main TestCipherMode 
...
[2024-10-15T22:15:02.584Z] jdk_security2_1_FAILED

Test_openjdk17_j9_sanity.openjdk_x86-64_windows_Personal: Known issue https://github.com/eclipse-openj9/openj9/issues/18708

[2024-10-16T01:51:10.631Z] Caused by: java.lang.OutOfMemoryError: Java heap space
[2024-10-16T01:51:10.631Z] 	at java.base/java.lang.String.repeat(String.java:5202)
...
[2024-10-16T02:02:18.562Z] jdk_util_0_FAILED

Test_openjdk8_j9_sanity.functional_x86-64_linux_Personal: Known issue: https://github.com/eclipse-openj9/openj9/issues/19224

[2024-10-15T22:44:29.181Z]  [ERR] /home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_x86-64_linux_Personal_testList_1/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/jitserver/jitserverconfig.sh: line 30: lsof: command not found
...
[2024-10-15T22:44:29.181Z] testJitserverArguments_0_FAILED

Valhalla test passes on x86-64 which is what I have tested locally. Valhalla on other platforms (Power, Z, aarch64) fail, which I will take a look at next.

I'm not sure about vt_standard tests yet

[hzongaro]

I took a quick look at the sanity.functional and sanity.openjdk "vt-standard" testing. It looks like those tests have seen many failures in recent runs. For instance, an aarch64 Linux sanity.functional vt-standard PR test run from earlier this month yielded 55 failures.

We'll need to do further investigation, but it seems unlikely that this pull request is responsible for the failures. These failures will not hold back merging this pull request.

• JDKNext aarch64 Linux sanity.functional vt-standard
• JDKNext PPC64LE Linux sanity.functional vt-standard
• JDKNext S390x Linux sanity.functional vt-standard
• JDKNext x86-64 Linux sanity.functional vt-standard
• JDKNext aarch64 Linux sanity.openjdk vt-standard
• JDKNext PPC64LE Linux sanity.openjdk vt-standard
• JDKNext S390x Linux sanity.openjdk vt-standard
• JDKNext x86-64 Linux sanity.openjdk vt-standard

[a7ehuo]

Valhalla test passes on x86-64 which is what I have tested locally. Valhalla on other platforms (Power, Z, aarch64) fail, which I will take a look at next.

Two main tests fail on Power, Z, and Aarch64: ValueTypeArrayTestsJIT and ValueTypeSystemArraycopyTests.

I looked at the failure in ValueTypeArrayTestsJIT first. The failure is related to how instanceOf is evaluated on Power, Z, and Aarch64. On x86, we call helper jitInstanceOf which explains why it passes.

It failed because instanceOf doesn't think null restricted array PointPV[] is an instance of [Lorg/openj9/test/lworld/ValueTypeArrayTests$PointPV; in ValueTypeArrayTests.runTest

[ValueTypeArrayTests] [INFO] calling runTest testArrays[3] NullRestrictedArray nullObj kinds[0] OBJ_TYPE 1 kinds[0] OBJ_TYPE 1
[ValueTypeArrayTests] [INFO] --------------------------
[ValueTypeArrayTests] [INFO] runTest staticArrayKind=OBJ_TYPE 1 staticSourceKind=OBJ_TYPE 1
[ValueTypeArrayTests] [INFO] runTest arr: actualArrayKind=IFACE_TYPE 2 sourceVal: actualSourceKind=NULL_REF 0 //<--- wrong actualArrayKind should be PRIM_TYPE 4 since testArrays[3] is NullRestrictedArray
[ValueTypeArrayTests] [INFO] runTest expectedExceptionClass null
[ValueTypeArrayTests] [INFO] assignDispatch arrKind=OBJ_TYPE srcKind=OBJ_TYPE idx=1
[ValueTypeArrayTests] [INFO] runTest caughtThrowable=java.lang.ArrayStoreException

[hzongaro]

I looked at the failure in ValueTypeArrayTestsJIT first. The failure is related to how instanceOf is evaluated on Power, Z, and Aarch64. On x86, we call helper jitInstanceOf which explains why it passes.

It failed because instanceOf doesn't think null restricted array PointPV[] is an instance of [Lorg/openj9/test/lworld/ValueTypeArrayTests$PointPV; in ValueTypeArrayTests.runTest

Thanks, @a7ehuo! Should the relevant tests still be disabled and an issue opened, or do you plan to try to fix this problem in this pull request?

[a7ehuo]

I plan to disable the related two tests (ValueTypeArrayTestsJIT, ValueTypeSystemArraycopyTests) for now and open new PRs when ready.

There are already issues opened on them. I think I will reuse them for future submission. I will make a note in these two issues that this PR doesn't fix them on all hardware. https://github.com/eclipse-openj9/openj9/issues/19913 https://github.com/eclipse-openj9/openj9/issues/19914

I'll run some Jenkins Valhalla tests on all platforms before pushing another commit

[a7ehuo]

@hzongaro I reverted test/functional/Valhalla/playlist.xml to baseline and tested internally that the Valhalla functional tests pass on all platforms. It's ready for another review. Thanks you!

[hzongaro]

Jenkins test sanity.functional,extended.functional xlinuxval,plinuxval,zlinuxval,alinuxval jdknext depends eclipse/omr#7452

[hzongaro]

Jenkins test sanity.functional,sanity.openjdk all jdk8,jdk11,jdk17,jdk21 depends eclipse/omr#7452

[hzongaro]

Jenkins test sanity.functional,sanity.openjdk xmac jdk11 depends eclipse/omr#7452

[a7ehuo]

Checked three failed tests and all known issues (details as below).

Waiting for openjdk11_j9_sanity.functional_x86-64_mac to complete

Test_openjdk21_j9_sanity.openjdk_x86-64_mac: Known issue: https://github.com/eclipse-openj9/openj9/issues/19987

[2024-10-21T02:28:49.205Z] STDERR:
[2024-10-21T02:28:49.205Z] javax.security.auth.login.LoginException: Unable to obtain Principal Name for authentication 
[2024-10-21T02:28:49.205Z] 	at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:818)
…

[2024-10-21T02:29:13.877Z] -----------------------------------
[2024-10-21T02:29:13.877Z] jdk_security4_1_FAILED
[2024-10-21T02:29:13.877Z] -----------------------------------

Test_openjdk8_j9_sanity.functional_x86-64_linux: Known issue: https://github.com/eclipse-openj9/openj9/issues/18599

[2024-10-21T04:03:57.367Z]  [ERR] /home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_x86-64_linux_Personal_testList_1/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/jitserver/jitserverconfig.sh: line 30: lsof: command not found
[2024-10-21T04:03:57.367Z]  [ERR] /home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_x86-64_linux_Personal_testList_1/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/jitserver/jitserverconfig.sh: line 30: lsof: command not found
[2024-10-21T04:03:57.367Z]  [ERR] Generating RSA private key, 2048 bit long modulus
…
[2024-10-21T04:03:57.367Z]  [ERR] failed to configure SSL ecdh: Success
[2024-10-21T04:03:57.367Z]  [ERR] Failed to initialize the SSL context
[2024-10-21T04:03:57.367Z]  [ERR] 
[2024-10-21T04:03:57.367Z]  [ERR] JITServer is ready to accept incoming requests
…

[2024-10-21T04:04:04.002Z] -----------------------------------
[2024-10-21T04:04:04.002Z] testJitserverArguments_0_FAILED
[2024-10-21T04:04:04.002Z] -----------------------------------

Test_openjdk8_j9_sanity.openjdk_s390x_linux: Known issue: https://github.com/eclipse-openj9/openj9/issues/14948#issuecomment-1206441095

[2024-10-21T03:27:55.722Z] Caused by: java.io.IOException: write beyond end of stream
[2024-10-21T03:27:55.722Z] 	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:201)
[2024-10-21T03:27:55.722Z] 	at CloseInflaterDeflaterTest.lambda$testDeflaterOutputStream$4(CloseInflaterDeflaterTest.java:153)
[2024-10-21T03:27:55.722Z] 	at org.testng.Assert.expectThrows(Assert.java:1005)
[2024-10-21T03:27:55.722Z] 	... 30 more
[2024-10-21T03:27:55.722Z] test CloseInflaterDeflaterTest.testDeflaterOutputStream(false): failure
…
[2024-10-21T03:27:55.723Z] Caused by: java.io.IOException: write beyond end of stream
[2024-10-21T03:27:55.723Z] 	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:201)
[2024-10-21T03:27:55.723Z] 	at CloseInflaterDeflaterTest.lambda$testDeflaterOutputStream$4(CloseInflaterDeflaterTest.java:153)
[2024-10-21T03:27:55.723Z] 	at org.testng.Assert.expectThrows(Assert.java:1005)
[2024-10-21T03:27:55.723Z] 	... 30 more
[2024-10-21T03:27:55.723Z] test CloseInflaterDeflaterTest.testGZip(true): failure
…
[2024-10-21T03:27:55.723Z] test CloseInflaterDeflaterTest.testZipCloseEntry(java.util.zip.ZipOutputStream@108de3fe): failure
[2024-10-21T03:27:55.723Z] java.io.IOException
[2024-10-21T03:27:55.723Z] 	at CloseInflaterDeflaterTest$1.write(CloseInflaterDeflaterTest.java:54)
…
[2024-10-21T03:28:01.017Z] -----------------------------------
[2024-10-21T03:28:01.017Z] jdk_util_1_FAILED
[2024-10-21T03:28:01.017Z] -----------------------------------

[a7ehuo]

Test_openjdk11_j9_sanity.functional_x86-64_mac has passed

[hzongaro]

Failures were due to known issues. OMR pull request eclipse/omr#7452 that this change depends upon has been merged and promoted. Merging.