v12: maintenance process for v12.0.0

[nqb]

Describe the bug If we keep things as-is regarding maintenance builds, each night if there is new commits on a maintenance branch, we will rebuild Docker images and push them as a first step in scheduled pipelines.

Contrary to packages which are build, tested and upload only if all tests passed, Docker images will be push on registry even if tests failed.

I don't think it's critical because in order to pull new images for a maintenance, you need to install a new package which means that a new package has been successfully uploaded. However, nothing prevent a user to call manage-images.sh to pull new images from registry without installing a new package. It could be a problem because it would mean that uncontainerized services will run with a code different from containerized services.

A possible solution I see is:

1. build and push Docker images on your registry as a first step using a specific tag like: test-maintenance-12-0
2. use this specific tag during our tests
3. if all tests passed, run a job which will:
   1. pull all images built at step 1 and push them with final tag (i.e. maintenance-12.0) or
   2. rebuild images and push them with final tag (I would like to avoid this)

Let me know your thoughts on that.

[julsemaan]

I like this approach, we use something similar in Fingerbank by tagging the images with the BUILD ID (some auto-generated UUID from Google Cloud Build) and then once its over, we add the 'real' tag to the image (and also leave the BUILD ID tag for tracability)

[nqb]

As suggested by @julsemaan, we can use GitHub registry API to add final tag to images at step 3. Or perhaps docker command or kaniko.