Add JREs to `linux` packaging system (feature request)

[theofficialgman]

Now that https://github.com/adoptium/installer/issues/330 is officially closed, and requests for JREs on the DEB/RPM repos were not fulfilled, I have created a new issue to track this suggestion.

JREs support was requested or mentioned in these locations: https://github.com/adoptium/installer/issues/330#issue-962596954 https://github.com/adoptium/installer/issues/330#issuecomment-998270994 https://github.com/adoptium/installer/issues/330#issuecomment-1015175480

[karianna]

For those watching, linuxNew has been renamed to linux

[Jeeppler]

To have Linux packages available is awesome, but JREs are really missing. JREs builds are a lot smaller compared to JDK builds.

[ascheman]

Please assign this to me (@smlambert) as discussed.

[ascheman]

I have some questions wrt. to implementation options and would be happy about some feedback.

Please checkout my changes at https://github.com/adoptium/installer/compare/master...ascheman-adoptium:installer:feature/430-poc-1-debian-jre-src

I started with a first PoC for Debian as I can see at least two implementation options:

1. Make a copy of all current src packages, e.g., 8 -> 8-jre (as I did here)
2. Parameterize the package in the given srcs. It should be possible to perform a paramterized build (since it is mostly based on a large GNU makefile).

I see drawbacks with both approaches:

1. A lot of stuff is duplicated and needs to be maintained redundantly. Therefor the code is a little simpler/better to understand for people without a strong Debian/make background.
2. Over the time it would look a little bit like the codebase of the official Debian JDK/JRE build which makes maintenance a little harder. If this approach was used, it perhaps would be consequent to merge all different JDK/JRE package builds into a single directory instead of having one per JDK version.

If desired, I could provide a second PoC to show an implementation of the second variant. But probably the team already has a preference and could meet a decision how to proceed.

Besides this I do have some more questions, but this is the most important one. Let's concentrate on this decision before proceeding.

And, of course, I will refactor/clean up my changes before raising a final pull-request.

[smlambert]

FYI @sophia-guo @jerboaa (upon your returns from vacation on Jan 9th) - may I ask your input to Gerd's questions above please :)

[jerboaa]

I cannot really comment on the debian packaging approach as I have basically no experience there. Having said that, it seems fine to duplicate installer files into <version>-jre dirs and have it maintained there. I'd expect the list of installed alternatives to differ for JDK vs. JRE anyway. My $0.02.

[sophia-guo]

For debian probably @gdams is the right person to help. I have no experience about it.

[tellison]

I'll jump on the bandwagon of not being experienced in Debian best-practices, bit looking at the PoC changes, it is probably better to accept the duplication and keep it simple - since any changes will have to be understood by the folks here who are not Debian installer experts. If we are lucky to attract somebody who is willing to take ownership of this space then the decision may change and the code can be optimised further.

[ascheman]

Thanks for the feedback, @tellison !

Then I'll go ahead and will provide similar changes for all the other Linux repositories, starting with Debian.

[ascheman]

Thinking of it again, I finally suggest to put everything JRE-related into a completely parallel tree:

Existing

linux/
├── Jenkinsfile
...
├── jdk
│   ├── alpine
│   ├── build.gradle
│   ├── debian
│   ├── redhat
│   └── suse

New

linux/
...
├── jre
│   ├── alpine
│   ├── build.gradle
│   ├── debian
│   ├── redhat
│   └── suse

And have Gradle tasks packageJreDebian etc.

[ascheman]

Further testing of preliminary Debian JRE versions required by @Jeeppler (as offered in mercedes-benz/sechub#1794).

[Jeeppler]

For people, who might be interested to know why a Linux installer for JRE make sense.

Scenario 1: Multi-stage build

One scenario in a containerized world, is to use the Docker multi-stage build pattern. The idea is simple, first the application gets build in one container using the Java JDK (builder container). Once the build finished, the application build artifact gets copied into a new container (application container). The application container runs the application using Java JRE. This reduces the image size significantly.

One could argue, that the same can be achieved by using the jlink tool. While this is true, the jlink tool can optimize applications for size by creating a custom runtime image. However, the end result is not a "standardized" runtime and it might be harder to reason if something goes wrong. The JRE is a standardized and well tested runtime with a small footprint.

The second limitation is, that maybe a second tool written in Java is used for a specific task by the main application. The second tool is not written by the same author and is getting downloaded into the container. This changes everything, now jlink is not an option anymore. However, using the JRE is still possible and still reduces the size of the container.

Scenario 2: Base Image

Let's assume an operation team creates a base image (Container or VM) to run Java application. The base image can then be used by other teams. A further assumption, the team creating the base image does not know who will use their image. In this instance they cannot use jlink tool to create a custom runtime. Instead they need a universal small runtime to keep the base image small.

[Jeeppler]

@ascheman can you tell us for what distributions you will create JRE installer packages?

I know you are working on:

• Debian/Ubuntu
• Alpine

What other distributions will you work on: Fedora, RockyLinux, RedHat Enterprise Linux?

[ascheman]

What other distributions will you work on: Fedora, RockyLinux, RedHat Enterprise Linux?

@Jeeppler: I will finally provide Linux JREs for the very same distributions for which Temurin JDK packages are created, i.e.,

• [x] Alpine
• [x] Debian
• [x] Red Hat (might also work on CentOS and Fedora)
• [x] SuSE

[Jeeppler]

I built the Alpine image using the following command:

DOCKER_BUILDKIT=1 ./gradlew clean packageJreAlpine checkJreAlpine --parallel -PPRODUCT=temurin -PPRODUCT_VERSION=8 -PARCH=amd64

However, encountered an error message along the way:

>>> ERROR: temurin-8-jre*: libawt.so: path not found
        java-common
        java-cacerts
        ttf-dejavu
        so:libX11.so.6
        so:libXext.so.6
        so:libXi.so.6
        so:libXrender.so.1
        so:libXtst.so.6
        so:libasound.so.2
        so:libc.musl-x86_64.so.1
        so:libfreetype.so.6
        so:libgcc_s.so.1

Regardless, the build went through successfully.

I was able to install the package:

apk add --allow-untrusted temurin-8-jre-8.352.08-r0.apk

and run Java:

java -version
openjdk version "1.8.0_352"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_352-b08)
OpenJDK 64-Bit Server VM (Temurin)(build 25.352-b08, mixed mode)

The error message does not appear while building JRE for Java 11 for Alpine.

[Jeeppler]

After building temurin-17, two apk packages are in the linux/jre/alpine/build/ospackage folder:

$ ls -1
APKINDEX.tar.gz
temurin-17-17.0.5_p8-r0.apk
temurin-17-jre-17.0.5_p8-r0.apk

I am not sure what the purpose of temurin-17-17.0.5_p8-r0.apk is.

In addition, building temurin-19 created two packages as well.

[Jeeppler]

While building the alpine images I got the following error message a couple of times:

> Task :jre:alpine:checkJreAlpine FAILED

UnknownClass.JUnit Jupiter > UnknownClass.executionError FAILED
    org.junit.platform.commons.JUnitException at EngineExecutionOrchestrator.java:114
        Caused by: org.junit.platform.commons.JUnitException at HierarchicalTestEngine.java:57
            Caused by: java.util.concurrent.ExecutionException at ForkJoinTask.java:1006
                Caused by: java.lang.OutOfMemoryError at NativeConstructorAccessorImpl.java:-2
                    Caused by: java.lang.OutOfMemoryError at NativeConstructorAccessorImpl.java:-2
                        Caused by: java.lang.OutOfMemoryError at Arrays.java:3745

UnknownClass.JUnit Jupiter FAILED

After restarting/building it again it worked or failed again. After a few tries it usually works. However, building Temurin-19 always ended with this error.

My computer had enough free memory. My hypothesis it might have something to do with the underlying container which is used during the build.

[ascheman]

While building the alpine images I got the following error message a couple of times:

> Task :jre:alpine:checkJreAlpine FAILED

UnknownClass.JUnit Jupiter > UnknownClass.executionError FAILED
    org.junit.platform.commons.JUnitException at EngineExecutionOrchestrator.java:114
        Caused by: org.junit.platform.commons.JUnitException at HierarchicalTestEngine.java:57
            Caused by: java.util.concurrent.ExecutionException at ForkJoinTask.java:1006
                Caused by: java.lang.OutOfMemoryError at NativeConstructorAccessorImpl.java:-2
                    Caused by: java.lang.OutOfMemoryError at NativeConstructorAccessorImpl.java:-2
                        Caused by: java.lang.OutOfMemoryError at Arrays.java:3745

UnknownClass.JUnit Jupiter FAILED

After restarting/building it again it worked or failed again. After a few tries it usually works. However, building Temurin-19 always ended with this error.

My computer had enough free memory. My hypothesis it might have something to do with the underlying container which is used during the build.

This looks like an error from the Unit-Tests executed after building the .deb / .apk / .rpm. From the OOM-Exception I would guess that you didn't set the recommended environment variables:

export DOCKER_BUILDKIT=1
export _JAVA_OPTIONS="-Xmx4g"

I tend to always forget something like this when switching between different projects. You probably want to use direnv to automatically set the variables when entering the Adoptium file tree.

[ascheman]

I built the Alpine image using the following command: ... However, encountered an error message along the way:

>>> ERROR: temurin-8-jre*: libawt.so: path not found

...

Regardless, the build went through successfully.

I was able to install the package:

apk add --allow-untrusted temurin-8-jre-8.352.08-r0.apk

and run Java:

java -version
openjdk version "1.8.0_352"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_352-b08)
OpenJDK 64-Bit Server VM (Temurin)(build 25.352-b08, mixed mode)

The error message does not appear while building JRE for Java 11 for Alpine.

The Alpine JDKs/JREs are headless, ie., they do not support UIs with AWT (search for Alpine in the respective documentation).

Not sure why they contain references to the libawt shared object at all? Is this a bug @smlambert?

[ascheman]

After building temurin-17, two apk packages are in the linux/jre/alpine/build/ospackage folder:

$ ls -1
APKINDEX.tar.gz
temurin-17-17.0.5_p8-r0.apk
temurin-17-jre-17.0.5_p8-r0.apk

I am not sure what the purpose of temurin-17-17.0.5_p8-r0.apk is.

In addition, building temurin-19 created two packages as well.

I am not so deep into Alpine builds yet and mostly copy/pasted the JDK stuff. As far as I can see the second APK contains a sub package.

To my understanding this does not make much sense as sub packages are meant to be optional. But in this case the JDK/JRE is the core of the APK. Perhaps @gdams as the author (or at least committer) of the Alpine JDK stuff can explain this?

[Jeeppler]

The Alpine JDKs/JREs are headless, ie., they do not support UIs with AWT

Is there any reason, why they are not named: temurin-17-jre-headless-17.0.5_p8-r0.apk?

[ascheman]

The Alpine JDKs/JREs are headless, ie., they do not support UIs with AWT

Is there any reason, why they are not named: temurin-17-jre-headless-17.0.5_p8-r0.apk?

Well, they are based on the Alpine JRE tarballs, which do not contain headless in their names either. I tried to be consistent here. But it's a good question, as both (tarball and APK) could reflect their reduced functionality in their names -> @smlambert / @gdams?

[jerboaa]

Not sure why they contain references to the libawt shared object at all?

For a headless build you shouldn't see libawt_xawt et. al., but only libawt_headless. libawt is always there AFAIK.

[Jeeppler]

The packages from Alpine itself have headless in there name: e.g. openjdk17-jre-headless. It would make sense to name the temurin packages similarly.

[Jeeppler]

Debian Smoke Tests

What I tested so far is to check if the packages for product_version (Java version) 8, 11, 17 and 19 can be build and installed into a Debian environment.

Steps

1. Build the Debian package using:

   ./gradlew clean packageJreDebian checkJreDebian --parallel -PPRODUCT=temurin -PPRODUCT_VERSION=<version> -PARCH=amd64
   

2. Copied the build from linux/jre/debian/build/ospackage and put it in a folder with a Debian Dockerfile (using debian:11-slim).

3. Started the container. While building the container the Temurin Debian dep and adoptium-ca-certificates_1.0.1-1_all.deb get copied into the container. In addition, the following dependencies are installed for temurin:

   apt install --assume-yes java-common libasound2 libfontconfig1 libfreetype6 libx11-6 libxext6 libxi6 libxrender1 libxtst6 fonts-dejavu
   

   The adoptium ca-certificates package needs the following dependencies:

   apt install --assume-yes p11-kit ca-certificates
   

4. Install both via dpkg. For example, for Temurin-17 (Java)

   dpkg -i adoptium-ca-certificates_1.0.1-1_all.deb temurin-17-jre_17.0.5+8_amd64.deb
   

5. Run java -version to see what version is installed.

It worked for all packages:

• Temurin 8
• Temurin 11
• Temurin 17
• Temurin 19

without any problems.

Next, I will test Alpine and do a test with SecHub and the JRE packages to see whether SecHub works with JRE packages as well. The only version I cannot test is Temurin 8, because SecHub requires already Java 11 as a minimum version.

[Jeeppler]

@ascheman the Debian packages should work on Ubuntu as well, is that correct?

[Jeeppler]

Alpine Smoke Tests

Steps

1. Build

DOCKER_BUILDKIT=1 ./gradlew clean packageJreAlpine checkJreAlpine --parallel -PPRODUCT=temurin -PPRODUCT_VERSION=8 -PARCH=amd64

2. Copied the build from linux/jre/alpine/build/ospackage and put it in a folder with a Alpine Dockerfile (using alpine:3.17).

3. Install packages

apk add --allow-untrusted temurin-8-jre-8.352.08-r0.apk

4. Run java -version to see what version is installed.

Results

Worked for all JRE versions:

• Temurin 8
• Temurin 11
• Temurin 17
• Temurin 19

[ascheman]

The packages from Alpine itself have headless in there name: e.g. openjdk17-jre-headless. It would make sense to name the temurin packages similarly.

I am with you. But the naming here should be consistent across packaging the tarballs and APKs (for both JDK and JRE). We have already a long discussion here - which IMHO should concentrate on the overall JRE bundling for several distros. As the naming question is somehow independent from this concern, I suggest to separate it and raise a new issue to rename it in all relevant places for Alpine. I would think that the umbrella project https://github.com/adoptium/adoptium/ is the right place for such an issue.

BTW: @Jeeppler would you mind to join the Adoptium Slack? Smaller discussions might be easier over there (though I appreciate to document the major things in Github issues).

[ascheman]

@ascheman the Debian packages should work on Ubuntu as well, is that correct?

Yes, as Ubuntu is heavily based on Debian, the packages should be working there as well.

[Jeeppler]

Alpine smoke tests works well for:

• 8u362
• 11.0.18
• 17.0.6
• 19.0.2