Swich the base image to Ubuntu 22.04 LTS

[eitsupi]

Just as the release dates of R4.0.0 and Ubuntu 20.04 were only a few days apart, the release date of R4.2.0 could be right after Ubuntu 22.04. Immediately after the release of jammy, there is a possibility that RSPM binary installation, RStudio, etc. are not compatible with jammy, so we may need to modify the build system.

https://discourse.ubuntu.com/t/jammy-jellyfish-release-schedule/23906

$ docker run --rm -it rocker/r-ver Rscript -e "read.csv('/usr/share/distro-info/ubuntu.csv') |> tail(1)"
     version        codename series    created    release        eol eol.server
36 22.04 LTS Jammy Jellyfish  jammy 2021-10-14 2022-04-21 2027-04-21 2027-04-21
      eol.esm
36 2032-04-21

[cboettig]

Yup. Usually binaries compiled on the older system ought to run on the newer ones (though maybe Dirk can correct me on that); e.g. I believe in python world they build their 'universal' binary wheels by compiling with an ancient versions of linux. So I'd think most packages would still work; though whether we want to ship any containers that install 20.04-based binaries on a 22.04 platform is another question. iirc last time RSPM had a few month lag (and then there's the question of NVIDIA libraries, which I think were some 6 mo behind the release of 20.04 or so; though maybe different this time).

Of course, 20.04 has a lot more life in it. I do think we want to stick with the rocker-versioned tradition of running with the latest stable release, but it does raise the issues of precisely when we switch, and if we continue to support 20.04-flavored images. With the new automation and new buildkit approach to tags, I imagine that is somewhat easier? Providing the same stack on multiple base distros seems pretty standard for docker projects these days. Thoughts?

[eddelbuettel]

Usually binaries compiled on the older system ought to run on the newer ones (though maybe Dirk can correct me on that)

Yes they do, almost always (more below). For example, over in PPA land Michael prepares the 5000+ r-cran-* packages in his repo on the LTS cadence, and I while I generally prefer CRAN-from-source I have one laptop where I use our lovely BSPM setup with the c2d4u PPA and it is awesome and generally works without fail. Ditto in "my" CI setup over at r-ci where a while back I flipped from bspm optional to bspm on: it also just works for every CI run as one wants it to.

There can be (rare) exceptions. When I packaged R 4.1.2 earlier in the week, the r-base-core binary from 20.04 LTS would not install on 21.04 or 21.10 because of one 'pinned' shared library it wanted in a version not around for what I was running. Sometimes you just install it from the older release; here I just punted and resubmitted the source package to be rebuilt in 21.04 and 21.10 as well as it is a) quick and b) of general interest.

Also consider how commerical apps ship. I run rstudio, slack, zoom, chrome, ... here and they all run. They tend to be 'sumo builds' with their dependencies statically linked in to avoid the above problem but it demonstrates that yes we can.

So I would suggest to be pragmatic and, dunno, maybe use the 20.04 binaries for a month or so until 22.04 will have caught up. The service you depend on here but do not pay for or have a SLA for simply is not obligated to serve you binaries on April 21 so maybe it is safer to assume (just on good engineering grounds) that you won't have them. You can always flip the switch early if they are... (I do the same for r-ci and wait til BSPM has caught up before I flip the default switch.)

[eitsupi]

In the current build system, first set the rocker/r-ver base image to the latest Ubuntu LTS retroactively from the R release date. (ubuntu:focal or ubuntu:jammy) Next, try the CRAN mirror as follows to see if the binary installation URL of RSPM can be used, and if not, set the source installation URL which is irrelevant to OS version. This means that until the binary install URL for jammy is enabled, the source install URL is used, and when the binary install URL is enabled, a PR is created that automatically detects it and changes the CRAN env var.

https://github.com/rocker-org/rocker-versioned2/blob/a382d7da54d907e81efd007aa2c49f36197bbb1b/build/make-stacks.R#L29-L35

https://github.com/rocker-org/rocker-versioned2/blob/a382d7da54d907e81efd007aa2c49f36197bbb1b/build/make-stacks.R#L46-L53

The problem with this method is that it temporarily causes rocker/r-ver:latest to not use binary installation. This will definitely affect the downstream images.

I was thinking of using focal as the base image until the binary for jammy is available (if you can tolerate the base image switching), but is it better to set the URL for focal until the binary installation URL for jammy is activated?

Providing the same stack on multiple base distros seems pretty standard for docker projects these days. Thoughts?

It is also possible to build a focal-based image and a jammy-based image in parallel, but this does not prevent the above switching between source installation/binary installation and switching between base images, so it is not so different from switching somewhere from focal to jammy and providing only one of them.

there's the question of NVIDIA libraries, which I think were some 6 mo behind the release of 20.04 or so; though maybe different this time

CUDA on jammy may continue to fail to build until it is provided, but I think this is acceptable as it does not affect the build of other images.

[eitsupi]

Since the R 4.2.0 release date has been set for 2022-04-22, rocker/r-ver:4.2.0 will be ubuntu:jammy based.

[eitsupi]

I was worried about whether rocker_scripts would work properly in jammy, so I installed R4.2.0 myself and tried it.

FROM ubuntu:jammy

ENV R_VERSION=patched
ENV TERM=xterm
ENV R_HOME=/usr/local/lib/R
ENV TZ=Etc/UTC

COPY scripts/install_R_source.sh /rocker_scripts/install_R_source.sh

RUN /rocker_scripts/install_R_source.sh

ENV CRAN=https://cloud.r-project.org
ENV LANG=en_US.UTF-8

COPY scripts /rocker_scripts

RUN /rocker_scripts/setup_R.sh

CMD ["R"]

As a result, I found that rocker/rstudio:4.2.0 is not buildable at this time. (i.e., as it is in this situation, only rocker/r-ver:4.2.0 can be built immediately after the R4.2.0 release and not all subsequent images)

~~First, libgc1c2 does not exist for jammy. However, it appears that libgc1 can be installed instead. I have no idea what the significance of this library is, can it be changed or removed? @cboettig~~ FIxed by #408 https://github.com/rocker-org/rocker-versioned2/blob/fc4420447341567401180be3a568b5e3d169001d/scripts/install_rstudio.sh#L20

Second, the latest version of RStudio does not support jammy (rstudio/rstudio#10902). This will need to wait for upstream updates.

Rocker scripts I tested

• [x] install_R_source.sh
• [x] setup_R.sh
• [x] install_rstudio.sh (#435, #477)
• [x] install_pandoc.sh
• [x] install_tidyverse.sh
• [x] install_shiny_server.sh
• [x] install_verse.sh
• [x] install_geospatial.sh

[eddelbuettel]

That made me curious as I had never ever heard of libgc1c2 and on 21.10 apt tells me it is purely virtual anyway, and provided by libgc-dev. Maybe that is a better package to depend upon? Has the advantage of existing in all:

edd@brad:~$ apt-cache search libgc1c2
libgc-dev - conservative garbage collector for C (development)
edd@brad:~$ apt-cache policy libgc-dev
libgc-dev:
  Installed: (none)
  Candidate: 1:8.0.6-1.1build1
  Version table:
     1:8.0.6-1.1build1 500
        500 http://us.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
edd@brad:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu Jammy Jellyfish (development branch)
Release:        22.04
Codename:       jammy
edd@brad:~$ 

(This is on a laptop where I had / have some woes with graphics so updated early.)

[eitsupi]

Upon further investigation, I found that this libgc did not exist in https://github.com/rocker-org/rocker-versioned, but was added in the first commit of this repository.

https://github.com/rocker-org/rocker-versioned2/blob/2ce09967aa1c91f307ac1ddc7dd9b29c9c329026/scripts/install_rstudio.sh#L59

This does not seem to have anything to do with RStudio Server so it is better to remove it, I think. (RStudio Server can be installed without this library)

@noamross Any thoughts?

[noamross]

Thanks for investigating, @eitsupi! I have no knowledge of the origin of using libgc1c2 or why we have libgc* in the Rstudio install scripts. I suggest removing it and checking that we can build the stack through geospatial and binder.

[eddelbuettel]

Yep. It's very good of you to check and audit. Sometimes the package management system shows us a rough edge and we nudge it along (esp with packages technically outside of the distro as the RStudio ones) but in general it should just resolve "magically". So if something is odd-looking without a clear explanation, it may indeed be a good idea to nix it.

[eitsupi]

Thank you for your prompt reply.

I have recently added a test workflow for rocker scripts, so I will first make sure that this will also test install_geospatial.sh and install_binder.sh before submitting a PR to remove libgc1c2.

https://github.com/rocker-org/rocker-versioned2/blob/fc4420447341567401180be3a568b5e3d169001d/tests/rocker_scripts/Dockerfile#L1-L17

[eitsupi]

Note that, I have verified in the past that install_geospatial.sh works alone on rocker/r-ver. (#241)

[eitsupi]

@cboettig @noamross https://github.com/rstudio/rstudio/issues/10902#issuecomment-1098332026 It appears that the stable version of RStudio Server may not be able to be installed on ubuntu:jammy for the next few months.

This does not prevent the build of rocker/r-ver:4.2.0 based on ubuntu:jammy, but given the number of rocker/rstudio downloads, the inability to build rocker/rstudio:4.2.0 would have a very significant impact.

To avoid this impact, why not change this rule not to use the latest Ubuntu LTS?

Images will use the most recent LTS available at the time when the corresponding R version was released.

For example, selecting the most recent Ubuntu LTS after a certain period of time (e.g. 90 days) since release? This logic change can be accomplished by simply changing the following (Of course, each document needs to be modified as well)

https://github.com/rocker-org/rocker-versioned2/blob/9e8232717a5ae4175cb406f97a97365ddae39ef1/build/make-stacks.R#L444

[noamross]

I think that we could define a standard policy as "approximately 90 days, contingent on testing"

[eitsupi]

Since the base image selection logic has been updated by #424, the switch to ubuntu:jammy has been postponed until the version of R that will appear in 2022-07-20 or later.

https://github.com/rocker-org/rocker-versioned2/blob/a97bd9e6538a946e6029fcfd371d78feb9a26eaa/build/make-stacks.R#L444

[eitsupi]

Today, the base image of rocker/r-ver:devel has switched to ubuntu:jammy. (ubuntu:latest is now ubuntu:jammy) So rocker/rstudio:devel is failing to build.

https://github.com/rocker-org/rocker-versioned2/blob/a97bd9e6538a946e6029fcfd371d78feb9a26eaa/dockerfiles/r-ver_devel.Dockerfile#L1

[devarops]

Hi 👋🏾

Did you decide when you are going to switch to ubuntu:latest (22.04)? Are you going to wait 90 days or until R 4.3.0 is released?

Thanks,

[eitsupi]

@devarops Thanks for the good question.

Switch the base image to ubuntu:jammy for versions of R to be released on or after 2022-07-20 (i.e. 4.2.X). Base images for R versions released before 2022-07-19 (i.e. 4.1.3, 4.2.0) will remain ubuntu:focal and will not switch.

The only one using ubuntu:latest for the base image is rocker/r-ver:devel, which is built daily and has already switched to Ubuntu 22.04 LTS.

[devarops]

@eitsupi Thank you for your answer. And thank you all for your work. I really appreciate the project, and I now depend on it.

[eitsupi]

Note: RSPM seems to have started providing binaries for Ubuntu 22.04 a few days after jammy's release, and the images used in r-universe have already switched to ubuntu:jammy. https://github.com/r-universe-org/base-image

As a side note, RSPM did not provide binaries for R4.2.0 as of the day of the R4.2.0 release, so the build in this repository was just under the 6-hour limit.

[cboettig]

yes, things seem to be moving a bit faster this time around, maybe 90 days is too conservative.

I'm watching https://gitlab.com/nvidia/container-images/cuda/-/issues/157 to see about the nvidia base images, though that branch of the stack could be dealt with as a separate case.

[eitsupi]

Thank you for checking on the cuda base image.

Note that for now the base image specify only has the ability to replace the current 20.04 with 22.04. (e.g. nvidia/cuda:11.1.1-cudnn8-devel-ubuntu22.04) This means that if the jammy-based cuda11.1.1 image will not be published, the build will fail. (Of course, this does not affect the success or failure of other image builds.)

https://github.com/rocker-org/rocker-versioned2/blob/a3289e39d7dbde364e45a9cc679d54f915df2bde/build/make-stacks.R#L114-L123

[eitsupi]

@cboettig It seems that cuda images only have cuda 11.7+ for ubuntu jammy. https://hub.docker.com/r/nvidia/cuda/tags?page=1&name=22.04

Currently the version of cuda is fixed at 11.1.1 in this repository, but I am wondering if this version can be changed.

[cboettig]

Yup, I think we would want to bump up to 11.7 when we switch to Jammy.

We haven't outlined a precise policy on how we version CUDA. I don't think we want to become tied in to old toolkit versions. The situation is a bit nuanced, because generally the driver version on the host platform must be >= the version of the CUDA toolkit we pin in the docker image (see nvidia docs).

Since 20.04 release, Ubuntu packages CUDA drivers in the main repos, so host machines (cloud machines) running with the default Ubuntu repos only will have old CUDA drivers that might not run docker images with more recent cuda-toolkits. I'm not sure if that's a real concern though, I think most users tend to install more recent drivers from NVIDIA (or use derivative distros that continue to upgrade to the latest NVIDIA drivers, like PopOS!)

In a way, the limited selection in the base images makes this issue easier for us. It probably keeps things simpler if our focal-era images are cuda 11.1.1 and our jammy-era images bump to 11.7.

[cboettig]

:tada: