aqa-tests icon indicating copy to clipboard operation
aqa-tests copied to clipboard

Request to move jdk_security1, 2, 4 to the sanity bucket

Open pshipton opened this issue 3 years ago • 4 comments

I'm wondering if jdk_security1, 2, 4 can be moved to the sanity bucket, they are currently in extended. OpenJ9 recently had some regressions in some of these categories, which took longer to spot since extended only runs weekly (and we missed a week).

1,2, 4 don't take too long to run, see the numbers below. 3 does take too long to consider moving it.

At the same time we could change the configuration, for OpenJ9 at least, to run sanity.openjdk in two parallel jobs.

Times from xlinux jdk8.

25 - - jdk_security1_0 	3 min 33 sec	OK	No	No
26 - - jdk_security1_1 	3 min 41 sec	OK	No	No
27 - - jdk_security1_2  	27 ms	OK	Yes	No
28 - - jdk_security2_0 	3 min 21 sec	OK	No	No
29 - - jdk_security2_1 	2 min 26 sec	OK	No	No
30 - - jdk_security2_2  	24 ms	OK	Yes	No
31 - - jdk_security3_0 	42 min	NOT OK	No	No
32 - - jdk_security3_1 	41 min	NOT OK	No	No
33 - - jdk_security3_2  	24 ms	OK	Yes	No
34 - - jdk_security4_0 	5 min 46 sec	OK	No	No
35 - - jdk_security4_1 	5 min 24 sec	OK	No	No
36 - - jdk_security4_2  	20 ms	OK	Yes	No

jdk11

34 - - jdk_security1_0 	8 min 24 sec	OK	No	No
35 - - jdk_security1_1 	8 min 9 sec	OK	No	No
36 - - jdk_security1_2  	27 ms	OK	Yes	No
37 - - jdk_security2_0 	4 min 49 sec	OK	No	No
38 - - jdk_security2_1 	3 min 50 sec	OK	No	No
39 - - jdk_security2_2  	24 ms	OK	Yes	No
40 - - jdk_security3_0 	1 hr 0 min	OK	No	No
41 - - jdk_security3_1 	58 min	OK	No	No
42 - - jdk_security3_2  	19 ms	OK	Yes	No
43 - - jdk_security4_0 	9 min 20 sec	OK	No	No
44 - - jdk_security4_1 	9 min 18 sec	OK	No	No
45 - - jdk_security4_2  	21 ms	OK	Yes	No

jdk17

37 - - jdk_security1_0 	9 min 20 sec	NOT OK	No	No
38 - - jdk_security1_1 	9 min 45 sec	NOT OK	No	No
39 - - jdk_security1_2  	19 ms	OK	Yes	No
40 - - jdk_security2_0 	4 min 44 sec	NOT OK	No	No
41 - - jdk_security2_1 	5 min 42 sec	NOT OK	No	No
42 - - jdk_security2_2  	13 ms	OK	Yes	No
43 - - jdk_security3_2  	15 ms	OK	Yes	No
44 - - jdk_security4_0 	12 min	OK	No	No
45 - - jdk_security4_1 	12 min	OK	No	No
46 - - jdk_security4_2  	13 ms	OK	Yes	No

pshipton avatar Jul 14 '22 18:07 pshipton

I am up for it. With hotspot, jdk_security1, 2, 4 take similar time. https://ci.adoptopenjdk.net/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/79/tapResults/ https://ci.adoptopenjdk.net/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/79/tapResults/ https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_x86-64_linux/68/tapResults/

llxia avatar Jul 14 '22 18:07 llxia

Before we do this, I'd better run some testing to ensure these tests are passing reliably across platforms on the OpenJ9 machines.

pshipton avatar Jul 14 '22 18:07 pshipton

I will propose tagging this issue for Outreachy candidates to take on, if we are ready to do this reorganization.

smlambert avatar Oct 10 '22 14:10 smlambert

I never did any OpenJ9 testing, but I've started some on jdk8 now. If this works I'll try other versions as well. https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/1340/ - jdk8 - passed all but Windows https://openj9-jenkins.osuosl.org/job/Grinder/1353/ - jdk8 Windows 64-bit - passed https://openj9-jenkins.osuosl.org/job/Grinder/1354/ - jdk8 Windows 32-bit - passed https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/1355/ - jdk11 - as per https://github.com/eclipse-openj9/openj9/issues/13756#issuecomment-1009455028, sun/security/krb5/auto/ReplayCacheTestProc.java is failing on xlinux (cent6-x64-5). I'm guessing we need to figure out how to update the machines to support his test. https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/1364/ - jdk17 - found wrong OS xlinux machine, passed other platforms https://openj9-jenkins.osuosl.org/job/Grinder/1373/ - jdk17 xlinux - passed (ub16)

pshipton avatar Oct 11 '22 15:10 pshipton