WIP: Added build & push to GH container registry & DockerHub

[KoalaGeo]

Overview

Same as https://github.com/geopython/pycsw/pull/714

I've added a GitHub workflow which builds a container and pushes this to the GitHub container registry. This is a useful parallel container delivery system to Dockerhub,

I've also updated the installation.rst and running-with-docker.rst to reference this option - docker pull ghcr.io/geopython/pygeoapi:latest (this will only work after PR)

I didn't remove the dockerhub related sections, but after a test period the ghcr.io/geopython/pygeoapi could be added to all examples, and potentially the Dockerhub workflow deprecated.

See & test container - https://github.com/KoalaGeo/pygeoapi/pkgs/container/pygeoapi

This does also associate packages with the whole organisation so they're visible at https://github.com/KoalaGeo?tab=packages or they would be at https://github.com/geopython?tab=packages

[kalxas]

Thank you @KoalaGeo

[KoalaGeo]

Hi @justb4,

Thanks for the feedback, I'll take a look & see what I can do

[justb4]

@KoalaGeo let me know if you need help. I am mainly familiar with setting up DockerHub build/push for GH Workflows. See thus the GeoHealthCheck example which uses the quite recent Action module docker/setup-buildx-action@v1.

Testing: I think one can make a Workflow dependent on another workflow: https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows#workflow_run so only if the main.yml is successful would the Docker build proceed. Also the pygeoapi Dockerfile contains the basic tests and data, so those tests could be run using the Docker image after it is built.

[KoalaGeo]

Hi @kalxas & @justb4,

I've got the conditions working nicely, build runs:

on: 
  push:
    branches:
      - master
    paths-ignore:
      - '**.md'  
  pull_request:
    branches:
      - master
    paths-ignore:
      - '**.md'  
  release:
    types:
      - released

and the docker build and push to GitHub Container Registry and DockerHub runs:

on:
  workflow_run:
    workflows: ["Build"]
    types: [completed]

So then bigger question (maybe for steering committee)

Do you want images on:

1. GHCR & DockerHub
2. GHCR Only
3. DockerHub only

?? I'd have a preference for 2, as I've had issues hitting pull limits on DockerHub, but pygeoapi has the open source waiver it's not an issue, I also aesthetically like that it's then all on one platform :-D

For the DockerHub workflow someone will need to add ENV variables to the repo for it to work: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }}

[KoalaGeo]

DockerHub workflow looks good but I can't test as geopython/pygeoapi is hard coded, I'd need to make it koalageo/pygeoapi to test then revert

[KoalaGeo]

Changing to my personal namespace worked - https://hub.docker.com/r/koalageo/pygeoapi

I've reverted target DockerHub repo to geopython/pygeoapi

[tomkralidis]

Any update on this issue?

[justb4]

See https://gitter.im/geopython/pygeoapi?at=61c1cbb1d40eb146b0e3a499 . So we need a vote.

[tomkralidis]

Thanks @justb4.

@justb4 @francbartoli @pvgenuchten @jorgejesus @kalxas @GeoSander what are your thoughts here on pygeoapi's Docker image publication:

1. GHCR & DockerHub
2. GHCR Only
3. DockerHub only
4. Other options?

[KoalaGeo]

@tomkralidis Sorry, Xmas and COVID working it's way through the family.

I was thinking from @justb4 review the workflow would build both dockerhub & ghcr, but the docs would refer to both existing but give examples re dockerhub to keep them readable/avoid duplication.

The ghcr adds a little bit of redundency and has limited downsides that I can see.

Also orgs that have issues hitting dockerhub limits could potentially use ghcr.

Happy to write whatever the team decide

[justb4]

To clarify from my review: what I think is important is the following workflow-step order for the Docker Image:

• build the Docker Image ONLY ONCE
• run the unit tests with this Image
• publish IMage to either GHCR and/or DockerHub

Only if first two steps are successful Docker push is started. In case of 2 Docker target repo's, doing separate builds/pushes (YAML files) can create inconsistencies between Docker images (think of a GH commit/push while one is building).

Under these conditions (steps) I am in favour of option 1 (GHCR and DockerHub).

[francbartoli]

+1 for option one

[tomkralidis]

@KoalaGeo let's go with option 1 with the workflow proposed by @justb4.

[justb4]

Keep up the good work @KoalaGeo ! Hope we all can help you. Tough material.

Had an interesting find trying to run unit tests with Docker Container: #848 . Running tests with 100% success would be required within the containers.yml workflow as a step between build and push. For some tests the issue is more general as I see many commented out in main.yml workflow..

[KoalaGeo]

Getting there, using https://github.com/docker/metadata-action which wasn't available before to add tags to the images as it should be a neater solution than previous .

However having issue getting it to pick up tags, when a release is made

[webb-ben]

Realizing https://github.com/geopython/pygeoapi/pull/866 is a duplicate of this. Just want to make whatever platform the pygeoapi image is published on offers both ARM64 and AMD64 architectures :)

[justb4]

I just revisited and this PR looks quite complete, it addresses the three conditions I earlier outlined:

• only run when main.yml (main Build) successful
• build for both DockerHub and GHCR
• push only when tests run with Docker passing

Though my knowledge of GHCR build/push is lacking. And the ARM64 and AMD64 architectures may be missing, see #866.

Proof will be in the pudding, once we run as geopython/pygeoapi...

[KoalaGeo]

Had a look again and this should be doing what you'd like @justb4

It runs only when Build is Complete Think I've got 1 build Then it runs the tests Then it pushes to docjkerhub & the GHCR.

If you accept the PR need to set: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} DockerHub repo: ${{ secrets.DOCKER_REPOSITORY }}