Jenkins-ci upgrade

[jburel]

PHASE I (Jan 2022) Current build is failing due to old version of Jenkins docker image. Main goal of this work is to have an updated working environment to test scl python 3.8 on CentOS 7

Several repositories need to be updated/adjusted in order to get things to work. Below is the list of repositories with changes:

Infra/Build

• Update devspace to work with jenkins lts and bump versions see https://github.com/ome/devspace/pull/190
• Update devslave-c7-docker see https://github.com/ome/devslave-c7-docker/pull/21 merged
• Fix issue in omero-install https://github.com/ome/omero-install/pull/261

DNS assumptions

• python-superbuild See
  • https://github.com/ome/omero-py/pull/316
  • https://github.com/ome/omero-python-superbuild/pull/135
• omero-build See https://github.com/ome/openmicroscopy/pull/6301

Jobs

• ~OMERO-gradle-plugins-build submodule https://github.com/ome/omero-gradle-plugins/pull/28~
• OMERO-Build: Fix build https://github.com/ome/openmicroscopy/pull/6302

Three jobs were green in merge-ci and failed on a fresh installation, giving us possible false positives.

• BIOFORMATS-image: PR skipping test for now https://github.com/ome/bio-formats-build/pull/241. Failing tests due to upgrade in Java 8 docker base image. Re-added flag preventing the failures
• BIOFORMATS-linkcheck (no work done)
• OMERO-training: See PR https://github.com/ome/openmicroscopy/pull/6303 merged

Others Other changes in BioFormats related repositories will also be required.

• The usage of older plugin prevents us from using the new way to push to nexus when using mvn deploy.
• ZarrReader prevents the usage of the more suitable flag altSnapshotDeploymentRepository.

PHASE II (Jan 2023)

Upgrade of Jenkins. Upgrade to 2.375.

• Upgrade to Java 11 (impact)
• Upgrade to Gradle 6 (impact)
• Upgrade to nodejs 1 (no impact)
• Postgresql 13

Infrastructure

• Java 11 + Gradle 6: https://github.com/ome/devspace/pull/199
• Node 16: https://github.com/ome/devspace/pull/200
• nginx 1.23: https://github.com/ome/devspace/pull/202
• Postgresql 13: https://github.com/ome/devspace/pull/201

Java 11 related issues: The following PRs fails the build due to a Javadoc related flag.

• https://github.com/ome/omero-artifact-plugin/pull/21 included in https://github.com/ome/omero-artifact-plugin/releases/tag/v5.6.0
• https://github.com/ome/omero-model/pull/90
• https://github.com/ome/omero-common/pull/39
• https://github.com/ome/omero-renderer/pull/23
• https://github.com/ome/omero-romio/pull/41
• https://github.com/ome/omero-blitz/pull/131
• https://github.com/ome/omero-server/pull/151
• https://github.com/ome/omero-gateway-java/pull/69

Gradle related issues:

• https://github.com/ome/omero-common/pull/40

[jburel]

Python test failures with Scl Python 3.8 https://github.com/ome/openmicroscopy/pull/6340

[jburel]

Second upgrade of Jenkins. Upgrade to 2.375. This version of Jenkins uses Java 11

Java 11 related issue:

• https://github.com/ome/omero-artifact-plugin/pull/21 included in https://github.com/ome/omero-artifact-plugin/releases/tag/v5.6.0

[sbesson]

Bio-Formats repository

The following minimal changes were made in order to run the repository tests in the staging devspace:

• the environment variables https://github.com/ome/devspace/blob/a732e08b79447be3b78d7a59a0c762d5c60a6461/.env#L22-L24 were set to production values, respectively /uod/idr/repos/curated and /uod/idr-scratch/devspace-jm/config
• the DATA_REPO_CONFIG-merge job was copied from https://merge-ci.openmicroscopy.org/jenkins/ to create a copy of the private configuration folder under REPO_CONFIG which can be re-used from the job and executed
• the volumes of the testintegration service in docker-compose.yml were reviewed to include /uod/idr-scratch:/uod/idr-scratch (and /uod/idr:/uod/idr:ro) and the services restarted

With this set of changes, a full execution of BIOFORMATS-test-repo is close to completion and matches the results of https://merge-ci.openmicroscopy.org/jenkins/.

The steps above expose a few elements which are currently absent from this repository:

• the configuration of the jobs working against private repositories. While this is not a requirement for building Bio-Formats/OMERO components, it is critical for executing the repository tests against the curated QA repository
• the convergence towards separating the application, under /scratch/<devspace_name> from the persistent/shared data under /uod/idr-scratch/<devspace_name>. This was initially explored in the context of merge-ci and the binary repository of the CI OMERO server currently lives there. The steps above would also proposal to migrate the shared configuration repository and there were some attempts to migrate nexus-data there, unsuccessful so far. At the administration level, this has the advantage of separating components which are growing at different rates and making the application way less sensible to out of disk errors. Doing so requires additional configuration steps, including symlinks as well as volume mounting as appropriate

More generally, the two points above raise the question of whether this repository should remain generic and act as a re-usable template or whether we should move towards capturing the state of the production deployment on the OME hardware.

[jburel]

mv ~/.ssh/known_hosts ~/.ssh/known_hosts.bak
ssh -T [email protected]

[sbesson]

A more programmatic option might be to set up the public SSH key fingerprints documented in https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints.