Run Docker images in an optimized way

[hoh]

We would like to start programs within containers inside on-demand VMs as fast as possible by preloading the containers on the filesystem of the VM.

However it like preloading Docker images is more difficult than I expected 🫤, at least using Docker - Podman may or may not be easier.

My attemps at mixing read-only and read-write in /var/lib/docker failed, mostly with invalid cross-device link

I tried something like this:

(run dockerd)
docker pull nginx

mksquashfs /var/lib/docker /root/docker.squashfs
mkdir /mnt/docker
mount /root/docker.squashfs /mnt/docker

mkdir /root/overlay
mount /dev/vdd /mnt/overlay (writable volume on the host)
mkdir /mnt/overlay/work /mnt/overlay/upper

mount overlay -t overlay -o lowerdir=/mnt/docker,upperdir=/root/overlay/upper,workdir=/root/overlay/work /var/lib/docker

[hoh]

After discussions, we may want to deploy a local image registry on the CRNs, and have the init1 fetch the images from that registry on start. This would prevent the duplication of storing the image both in tar and in extracted form.

A persistent volume for /var/lib/docker should always be present when using containers in a VM.

Inside the program JSON definition, we may want to add a field similar to the following:

        "containers": [
            "docker.io/nextcloud:25",
            "docker.io/postgres:13",
            {
                "item_hash": "7eb2eca2378ea8855336ed76c8b26219f1cb90234d04441de9cf8cb1c649d003",
                "image": "myapp:latest"
            }
        ],