feat: add capability to run installer per jdk version or distro selection

[zdtsw]

        - add  support ARCH with selection but not include "all" 
        - remove support VERSION with value "all"
        - make DISTRO to only "all" "Debian" and "RPM"
        - pass in jdkversion as variable for rpm
        -  default use node pool "dockerBuild&&linux&&x64", debian non x64 use pool "docker&&linux&&${ARCH}"
        - change from matrix to three different stages per each distro
        - only do git clone "installer.git" once and stash folder "linux" into different distro build stage
        - remove rpm for Centos8 publish to jfrog (already have rocky8)
        - add rpm for rhel6 to jfrog
        - add support in jenkins for 
           -  enableDebug for more build info
           -  split UPLOAD into "uploadPackage" and "uploadSRCRPM"
           -  can use the same job for test repo or branch (for debug purpose)
           -  add simple description of arch and distro per each job history
           -  set timeout to 2hr
           - 

       - add more test coverage:
          - missing test on oraclelinux8, ubi8
          - previous in one batch of bulding all RedHat/Suse RPMs, only test on x86_64
          - now at least one testcase per each arch, if rpm exists after build and keep "yum/dnf/zypper install" only on x86_64(same as before) due to docker host is for x64 only

Test run logs: RedHat arm32: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/125/console

RedHat x64: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/129/console

Suse x86: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/127/console

Suse aarch64 https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/128/console

Suse all https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/136/console

RedHat all https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/139/console

Debian x64 https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/138/console

Debian arm64 https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/140/console

Ref: https://github.com/adoptium/installer/issues/459

[zdtsw]

I'm confused. Current jenkins does support per jdk version or distro. What's the goal of this PR?

#459 requests for supporting per platform.

Thanks for pointing it out and I did a revisit of this issue. Did a test on existing Jenkinsfile(since I still do not have permission to access "release" in Jenkins), by using the current settings, we support runs on combination of

1. choose jdk version + DISTRO as RedHat
   • buildAndTest(RedHat, x64) which builds for all arch (except for i386/x86_32) with signing and test
2. choose jdk version + DISTRO as Suse
   • buildAndTest(Suse, x64) which builds for all arch (except for i386/x86_32) with signing and test
3. choose jdk version + DISTRO as Debian
   • buildAndTest(Debian, x64) with test
   • buildAndTest(Debian, s390x) except for jdk8 skip tests
   • buildAndTest(Debian, ppc64le) skip tests
   • buildAndTest(Debian, aarch64) skip tests
   • buildAndTest(Debian, armv7l) skip tests

The requirement here is to e.g: be able to make installer on RedHat s390x on JDK18 without waiting for others (i.e ppc64le,armv7l, aarch64) passed TCK compliance or AQA triage.

But I agree, implementation i made here, wont really help for this requirement. so I need to re-work on it.

[zdtsw]

if we are ready to run check task on non-X86 for Debian, we can take away the check on def gBuildTask = (buildArch == 'x86_64') ? "packageJdk${DISTRO} checkJdk${DISTRO}" : "packageJdk${DISTRO}"

[zdtsw]

it would look something like before:

after

[sxa]

@zdtsw What is the status of this PR? If it's not suitable to be merged in its current state can it be moved into draft?

[zdtsw]

@zdtsw What is the status of this PR? If it's not suitable to be merged in its current state can it be moved into draft?

I've requested review on the last commit, but haven't got any comments on it.

[karianna]

@sophia-guo left to review

[sophia-guo]

Could you @zdtsw also update the PR title to correctly describe what this PR addresses?

Since the main change is with jenkinsfile the github action workflow yml may not test it ( also workflow is only for linux-x64) I would suggest you run a jenkins job to test different platforms with different distro. Feel free to use https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/ without choose upload.

[sophia-guo]

While support per platform per version could we still keep the feature of support all platforms and versions? I think it's still useful especially when something wrong we can quickly and easily regenerate and upload all installers.

[zdtsw]

While support per platform per version could we still keep the feature of support all platforms and versions? I think it's still useful especially when something wrong we can quickly and easily regenerate and upload all installers.

I am thinking to keep both as two different jobs, it is up to the one doing release to see which fits better per each case.

[zdtsw]

Could you @zdtsw also update the PR title to correctly describe what this PR addresses?

Since the main change is with jenkinsfile the github action workflow yml may not test it ( also workflow is only for linux-x64) I would suggest you run a jenkins job to test different platforms with different distro. Feel free to use https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/ without choose upload.

thanks! will do

[zdtsw]

think i've fixed most of the issues mentioned in the previous comments: to only build jdk18 RH on x64: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/95/ to only build jdk17 deb on x64 https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/96/ to only build jdk11 Suse on x64 https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/99

still need to check why test failed on suse ppc64le https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/97 and test more combinations next week

[zdtsw]

by checking up the failed run, I realized the problem was how we design these tests in the past.

for RPM package test, it assumes that should be a x86_64.rpm available among other rpms. when this is changed (when i was test on suse + non-x86_64) test failed because we only generate a *.src.rpm + *.armv7hl.rpm therefore the problem was not only on Suse but both RPM jobs.

[zdtsw]

another change in the junit is to only test "yum/dnf/zypper install" when ARCH is set to x86_64 for another other arch, test covers if rpm package has been successfully built.

[zdtsw]

RedHat arm32: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/125/console

RedHat x64: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/129/console

Suse x86: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/127/console

Suse aarch64 https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/128/console

[zdtsw]

need another round of review on this PR

most of the big changes have been caught in the PR description.

and test result is updated with links too. GH action covers the ARCH=="all" scenario

[zdtsw]

@steelhead31 maybe you can do a review on the dpkg-buildpackage part ? with all these flags etc.