aqa-tests icon indicating copy to clipboard operation
aqa-tests copied to clipboard

Review of openjdk test targets

Open sophia-guo opened this issue 3 years ago • 4 comments

Most of openjdk test targets were created at the initial stage of aqa-tests. Some of them might need to be updated as test environment has been changed or test groups in upstream openjdk have updates.

[ ] targets limited to jdk8 and jdk9 - disabled for 11+ as originally no test images available

  • hotspot_serviceability
  • all target start with hotspot_

[ ] some testcases might be duplicate in multiple targets

  • jdk_svc_sanity and jdk_jdk https://github.com/adoptium/aqa-tests/issues/3993
  • jdk_svc_sanity and jdk_jfi https://github.com/adoptium/aqa-tests/issues/4013
  • jdk11_tier1_buffer and jdk_nio https://github.com/adoptium/aqa-tests/issues/3998
  • jdk11_tier1_iso8859 is subset of jdk_nio https://github.com/adoptium/aqa-tests/issues/4020
  • jdk11_tier1_pack200 is subset of jdk_tools https://github.com/adoptium/aqa-tests/issues/4021

[ ] any other targets?

  • https://github.com/adoptium/aqa-tests/issues/3991

sophia-guo avatar Oct 06 '22 19:10 sophia-guo

Note:

jdk11_tier1_buffer and jdk_nio https://github.com/adoptium/aqa-tests/issues/3998 jdk11_tier1_iso8859 is subset of jdk_nio https://github.com/adoptium/aqa-tests/issues/4020 jdk11_tier1_pack200 is subset of jdk_tools https://github.com/adoptium/aqa-tests/issues/4021

Those jdk**tier1** targets were originally setup to ensure all tier1 targets are covered by level sanity. So tests can be run nightly and if there is a failure it can be noticed immediately. All those tier1_** targets are subset of other targets of extended, which means there are duplicate testcases running in sanity and extended.

jdk_svc_sanity and jdk_jdk https://github.com/adoptium/aqa-tests/issues/3993 jdk_svc_sanity and jdk_jfi https://github.com/adoptium/aqa-tests/issues/4013

Those two are similar situation. Deleting those targets https://github.com/adoptium/aqa-tests/pull/4018 https://github.com/adoptium/aqa-tests/pull/3994 https://github.com/adoptium/aqa-tests/pull/4019 means moving those tier1 testcases from sanity to extended. Tests will only run weekly.

sophia-guo avatar Oct 11 '22 20:10 sophia-guo

Note : some sanity tests has been moved from sanity to extended https://github.com/adoptium/aqa-tests/pull/3994 https://github.com/adoptium/aqa-tests/pull/4019

When reorganize the tier1 and tier1 targets probably need to move those tests back.

Another issue is #4019 make sanity aix failed with no test selected. https://ci.adoptopenjdk.net/job/Test_openjdk11_hs_sanity.openjdk_ppc64_aix/530/

sophia-guo avatar Oct 19 '22 20:10 sophia-guo

https://github.com/adoptium/aqa-tests/issues/4218

sophia-guo avatar Jan 10 '23 16:01 sophia-guo

Related: https://github.com/adoptium/aqa-tests/issues/4127

smlambert avatar Aug 16 '23 02:08 smlambert