Build containers in GHA CI

[dschwoerer]

Working towards fixing #2186

Next step would be to also build on releases.

And of course this needs to be adjusted for the boutproject repository, I have been pushing images here

They are based on: https://github.com/dschwoerer/bout-container-base So petsc is included, but doesn't need to be rebuild each time.

[dschwoerer]

Also, probably should note, for this to work, the password for the docker account need to be added.

[ZedThree]

Maybe there's a dockerhub token we can add, similar to the PyPI one?

[bendudson]

Maybe there's a dockerhub token we can add, similar to the PyPI one?

That should be possible. I've got the dockerhub details somewhere and can add them to a Github secret or whatever it's called.

[dschwoerer]

We should probably switch to quay.io to avoid the duckerhub rate limiting ...

[ZedThree]

We could also just use Github's own package registry?

[dschwoerer]

No, we cannot. They have a limit of 500 MB for images - that is not even 1 image

[dschwoerer]

Ah, that is for private repositories, so we should be fine :sweat: Always gets me -.-

In that case, sure, we could use ghcr.io :+1:

[dschwoerer]

Now with ghcr.io :-)

Everybody can publish packages at https://github.com/boutproject/BOUT-dev/pkgs/container/bout-dev but deleting ones, e.g. with bad tags, can only be done by admins :-/

[ZedThree]

Awesome, thanks @dschwoerer! @tomc271 has also been looking at reviving the docker images -- Tom, can you have a look over this please?

[tomc271]

Location of mpiexec is not in the PATH environment variable; Need to prefix it with $MPI_BIN/ in order to use it. Should it be added to the PATH?

[dschwoerer]

It seems it is not possible to do that with ENV:

https://stackoverflow.com/q/46032023

However, as we would currently need to update it everytime a new python lands into fedora, it might be better to drop something in /etc/profiles.d/? I haven't tested whether that actually works, but I would assume that any shell should parse that at startup ...

[dschwoerer]

/etc/profiles.d/ is only read by login shells - so this is not useful /etc/environment seems also ignored. I now have something hopefully working with ENV: https://github.com/dschwoerer/bout-container-base/blob/main/Dockerfile#L28

However, openmpi build of PETSc seems stuck ...

[ZedThree]

Fedora builds now failing with:

 Problem: conflicting requests
  - nothing provides mpich-devel(x86-32) = 4.0.3 needed by petsc-mpich-devel-3.18.5-1.fc39.i686 from rawhide
  - nothing provides mpich-devel(x86-64) = 4.0.3 needed by petsc-mpich-devel-3.18.5-1.fc39.x86_64 from rawhide

[dschwoerer]

Reported here: https://bugzilla.redhat.com/show_bug.cgi?id=2215261 Workaround build in progress: https://copr.fedorainfracloud.org/coprs/davidsch/fixes4bout/packages/

As soon as the build is finished we should be able to rerun the failed tests.

[dschwoerer]

I noticed some unrelated cmake changes sneaked in - should I rebase to get them out?

[dschwoerer]

@tomc271 this is now fixed:

podman run --rm ghcr.io/boutproject/bout-dev:mpich-3d-opt-docker-ci mpirun --version
HYDRA build details:
[...]

[tomc271]

Should bout-container-base become a boutproject repository? (We already have https://github.com/boutproject/docker-bout)

[dschwoerer]

Should bout-container-base become a boutproject repository?

Probably, yes. I can do it if this gets merged.

(We already have https://github.com/boutproject/docker-bout)

This is probably not needed anymore, as it becomes part of BOUT++ with this PR.

[tomc271]

Testing the docker container:

The 'conduction' example built and ran fine (if a bit slow? 2m 12s)

However unable to find pip:

bash-5.2$ pip install xbout
bash: pip: command not found

[dschwoerer]

Which container did you use? Optimised?

Only some base images come with pip: https://github.com/dschwoerer/bout-container-base/blob/main/Dockerfile#L13

You can get pip with either: sudo dnf install python3-pip or with python -m ensurepip either should work ...

[tomc271]

Ah ok. It was mpich-3d-debug-docker-ci