rbe_ubuntu_20.04 based docker image?

[davido]

On 23.04.202, Ubuntu 20.04 was released: [1]. What is the ETA for the Ubuntu 20.04 based RBE image?

The reason I'm asking is that for Gerrit Code Review project we have intergration tests based on recent Git package release (to test Gerrit with Git wire protocol v2). And Git version is outdated in 16.04 and 18.04 Ubuntu releases.

Yes, I know that we could easily build our own custom RBE docker image and publish it somewhere. E.g. on https://gcr.io or on https://hub.docker.com registries.

• [1] https://wiki.ubuntu.com/FocalFossa/ReleaseNotes

[smukherj1]

We don't currently have plans for an Ubuntu 20.04 image. We would first need an Ubuntu 20.04 image to use as the base image from the managed base images maintained in https://github.com/GoogleContainerTools/base-images-docker.

After that, I'm open to PRs in https://github.com/GoogleCloudPlatform/layer-definitions and https://github.com/GoogleCloudPlatform/container-definitions adding the image.

[davido]

@smukherj1 Thanks for clarifying. I can look into sending a PR once the prerequisites are met.

[davido]

@philwo Are you working on it?

I also tried to add custom docker image from Bazel in Gerrit, in patch set 4 of this change: [1] but I have not managed to make it work. The error was:

/home/davido/.cache/bazel/_bazel_davido/27a001f4182820ef315d8d2d4f1edafe/external/remote_java_tools/BUILD:273:11: Compiling java_tools/src/main/cpp/util/file_posix.cc failed: (Exit 34): INVALID_ARGUMENT: Invalid arguments: 
  "command.ValidateSpec": Invalid spec - docker container must be specified

The reason I am asking: Gerrit switched to Java language level 11 recently, but unfortunately, RBE wasn't switched yet, so on gerrit@HEAD:

  $ bazel build --config=remote --remote_instance_name=<project> java/com/google/gerrit/common:server
 [...]
INFO: Found 1 target...
Target //java/com/google/gerrit/common:server up-to-date:
  bazel-bin/java/com/google/gerrit/common/libserver.jar
INFO: Elapsed time: 33.154s, Critical Path: 23.67s
INFO: 253 processes: 236 remote cache hit, 16 internal, 1 remote.
INFO: Build completed successfully, 253 total actions

  $ javap -v -cp bazel-bin/java/com/google/gerrit/common/libserver.jar com.google.gerrit.common.Version | grep major
  major version: 52

Local build works as expected and produce Java language level 11:

  $ bazel build java/com/google/gerrit/common:server
  [...]
INFO: Found 1 target...
Target //java/com/google/gerrit/common:server up-to-date:
  bazel-bin/java/com/google/gerrit/common/libserver.jar
INFO: Elapsed time: 7.942s, Critical Path: 5.34s
INFO: 253 processes: 236 remote cache hit, 16 internal, 1 linux-sandbox.
INFO: Build completed successfully, 253 total actions

  $ javap -v -cp bazel-bin/java/com/google/gerrit/common/libserver.jar com.google.gerrit.common.Version | grep major
  major version: 55

[1] https://gerrit-review.googlesource.com/c/gerrit/+/291864/4

[davido]

Update: I figured out how to build with JDK 11 and even produce major bytecode version 55 with RBE: [1].

[1] https://gerrit-review.googlesource.com/c/gerrit/+/291943

[davido]

Update: it seems that Ubuntu20.04 based docker image was added to: [1].

[1] https://github.com/GoogleContainerTools/base-images-docker/blob/master/ubuntu2004/

[davido]

I also figured, that we could build Ubuntu 20.04 compatible image by just saying:

Dockerfile

FROM gcr.io/cloud-marketplace/google/ubuntu2004:latest

RUN apt-get -y update && \
    apt-get -y install git && \
    apt-get -y install golang && \
    apt-get -y install openjdk-17-jdk openjdk-17-jre openjdk-17-jdk-headless && \
    apt-get -y install unzip zip && \
    apt-get -y install python3 && \
    apt-get -y install wget && \
    apt-get -y install vim && \
    apt-get clean

That way built Docker image works as expected locally (bazel build headless). I have not tried to use it on RBE, though.

[bduffany]

@smukherj1 I am interested in contributing to this -- one thing I don't understand is how to build ubuntu20.04-specific releases of clang. For example, here: https://github.com/GoogleCloudPlatform/layer-definitions/blob/ef7ab83fe8f7813ceeef5bb8dc283a08bed47151/layers/ubuntu1804/clang/deps.bzl#L29-L34

How is the object at ["https://storage.googleapis.com/clang-builds-stable/clang-ubuntu18_04/clang_r" + CLANG_INSTALLER.revision + ".tar.gz"] built and deployed?

More generally, since I will have the same questions for the Java layers, etc. -- is there any documentation that can be shared about how the source files in https://github.com/GoogleCloudPlatform/layer-definitions are generated?

[bduffany]

Friendly ping on this, since ubuntu 18.04 includes some dependencies that are out of date. Are you still open to PRs here? If so I would love to contribute, but could use some help with the questions in my above comment.

[steve-261370]

Just as another alternative solution here since this repo seems abandoned:

I found that bazel's own continuous build has built an ubuntu 20.04 image that you can use (and also a 22.04 one too).

The Dockerfile: https://github.com/bazelbuild/continuous-integration/blob/08fa4f8b1648acab1726e1354c3b56719cfba417/buildkite/docker/ubuntu2004/Dockerfile The custom cpp_env.json: https://github.com/bazelbuild/continuous-integration/blob/672777d1ae17d993a5ca3de3cb9a9e2fab6531ee/rbe-configs/cpp_env/ubuntu2004.json

Which you can use as such:

./rbe_configs_gen \
    --bazel_version=6.4.0 \
    --toolchain_container=gcr.io/bazel-public/ubuntu2004-java11@sha256:833b20afd7b624693160c8e9f8aafaee46657d29d32e61012256aa3d045d8662 \
    --output_src_root=/path/to/repo \
    --output_config_path=path/to/config-directory \
    --exec_os=linux \
    --target_os=linux \
    --cpp_env_json=ubuntu2004.json