build
build copied to clipboard
nodejs
Jenkins will require Java 11 from September LTS release
We probably need to start planning for this:
https://www.jenkins.io/blog/2022/06/28/require-java-11/
Beginning with Jenkins 2.357 (released on June 28, 2022) and the forthcoming September LTS release, Jenkins requires Java 11.
Both CI servers are currently running Java 8.
cc @nodejs/build-infra
Looking at the release schedules (https://github.com/nodejs/Release/issues/737, https://github.com/nodejs/Release/issues/658, https://github.com/nodejs/Release/issues/567) I'm going to plan to do the updates on 25 Aug 2022, with the proviso that if a security release is needed/happens during August that requires a CI lockdown I would shift doing the updates to between after the security release and before lifting the CI lockdown.
Maintenance has started. I've shutdown the test Jenkins CI, powered off the server and am in the process of taking a snapshot in DigitalOcean (infra-digitalocean-ubuntu14-x64-1-1661424784910).
As a (self-)reminder: just before shutting down the server I unselected all "build" (can start builds) permissions to prevent builds from being started during the updates and will need to put them back when the maintenance is over.
I've managed to upgrade the server from Ubuntu 16.04->18.04->20.04->22.04.1. Had a nameserver issue, which we've run into before with DO droplets and required the same fix as https://github.com/nodejs/build/pull/2745#issuecomment-911842863. Working on switching to Java 17 now.
Also the initial update to Ubuntu 18.04 still had the server on a very old kernel:
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 3.13.0-57-generic x86_64)
There's a "Kernel" page for the droplet in DigitalOcean which I switched to "Grubloader" and after a restart the server then used the expected kernel for the Ubuntu release. We're now on:
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-46-generic x86_64)
Have had to apply https://github.com/afonsof/jenkins-material-theme/issues/193. It looks like the theme were using has some issues with the latest LTS Jenkins release 😞 .
Run into an issue with Java 17 -- when I start Jenkins with it, attempting to start a build that uses our VersionSelectorScript.groovy is now erroring: e.g. https://ci.nodejs.org/job/node-test-commit-linuxone/33786/console
17:34:16 FATAL: For input string: "[49, 57]"
17:34:16 java.lang.NumberFormatException: For input string: "[49, 57]"
17:34:16 at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
17:34:16 at java.base/java.lang.Integer.parseInt(Integer.java:668)
17:34:16 at java.base/java.lang.Integer.valueOf(Integer.java:999)
17:34:16 at org.codehaus.groovy.runtime.StringGroovyMethods.toInteger(StringGroovyMethods.java:3319)
17:34:16 at org.codehaus.groovy.runtime.dgm$1162.invoke(Unknown Source)
17:34:16 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
17:34:16 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
17:34:16 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
17:34:16 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
17:34:16 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
17:34:16 at Script1.run(Script1.groovy:161)
17:34:16 at groovy.lang.GroovyShell.evaluate(GroovyShell.java:574)
17:34:16 at groovy.lang.GroovyShell.evaluate(GroovyShell.java:612)
17:34:16 at groovy.lang.GroovyShell.evaluate(GroovyShell.java:583)
17:34:16 at groovy.lang.Script.evaluate(Script.java:210)
17:34:16 at groovy.lang.Script$evaluate$0.callCurrent(Unknown Source)
17:34:16 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
17:34:16 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:157)
17:34:16 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:169)
17:34:16 at Script1.run(Script1.groovy:11)
49 and 57 are ASCII codes for "1" and "9", e.g. "19". Problem goes away (i.e. job successfully starts) if I switch back to Java 8 and restart Jenkins 😞 .
Nope, same error with Java 11 😞. https://ci.nodejs.org/job/node-test-commit-linuxone/33788/console
Looks like we're running into https://github.com/jenkinsci/matrix-groovy-execution-strategy-plugin/issues/20 😞 .
Looks like the current LTS version of Jenkins and the custom theme we have don't play well together -- black text on black background for console output in jobs and several missing icons. I've removed our custom theme for now.
Looks like we're running into jenkinsci/matrix-groovy-execution-strategy-plugin#20 😞 .
Implemented a workaround in https://github.com/nodejs/build/pull/3019. Needs a small fixup: https://github.com/nodejs/build/pull/3020
Summary:
- Test CI server has been updated from Ubuntu 16.04->18.04->20.04->22.04.1. Jenkins is now running on Java 17.
- Release CI server has been updated from Ubuntu 18.04->20.04. Jenkins is now running on Java 17. There was a disk space issue when attempting upgrading to Ubuntu 22.04 that I haven't had time to look into, but I have no immediate concerns about running on Ubuntu 20.04.
