Provide Docker/Podman/apptainer images.

[ghuls]

It would be nice to have zsync2 as Docker/Podman/apptainer images besides AppImage.

In an HPC environment, there is in general no option to install AppImage.

Singularity/Apptainer (http://apptainer.org/) and Podman (rootless Docker) is for example the only way to run containers on our HPC infrastructure. zsync2 is quite difficult to compile natively on old RedHat versions.

Docker/Podman and SingularityApptainer images are quite easy to make from the Apptainer image.

prepare_zsync2_rootfs.sh:

#!/bin/bash

# Ubuntu Bionic OCI core root image.
ubuntu_oci_root_image_url='https://partner-images.canonical.com/oci/bionic/current/ubuntu-bionic-oci-amd64-root.tar.gz'
ubuntu_oci_root_image_file='ubuntu-bionic-oci-amd64-root.tar.gz'

# Get version/commit from AppImage release links: https://github.com/AppImage/zsync2/releases
zsync2_version='2.0.0-alpha-1-20220602'
zsync2_commit='51-4e549b6'

zsync2_app_image_url="https://github.com/AppImage/zsync2/releases/download/${zsync2_version}/zsync2-${zsync2_commit}-x86_64.AppImage"
zsync2_app_image_file="zsync2-${zsync2_commit}-x86_64.AppImage"


# Download Ubuntu Bionic OCI core root image if not available or out of date.
if [ ! -e "${ubuntu_oci_root_image_file}" ] ; then
    curl -R -L -O "${ubuntu_oci_root_image_url}";
else
   # Only download Ubuntu OCI core root image, if there is a newer version.
   curl -R -L -O -z "${ubuntu_oci_root_image_file}" "${ubuntu_oci_root_image_url}";
fi

# Download zsync2 AppImage if not availabe.
if [ ! -e "${zsync2_app_image_file}" ] ; then
    curl -R -L -O "${zsync2_app_image_url}";
    
    chmod a+x "${zsync2_app_image_file}";
fi

rm -rf ./squashfs-root/

# Extract AppImage.
"./${zsync2_app_image_file}" --appimage-extract

# Fix symlink.
rm -r ./squashfs-root/usr/share/doc/libhogweed4
ln -s libnettle6 ./squashfs-root/usr/share/doc/libhogweed4

DockerFile

FROM scratch

ADD /squashfs-root/usr/ /usr
ADD ubuntu-bionic-oci-amd64-root.tar.gz /

CMD ["bash"]

zsync2.def : Singularity/Apptainer file.

Bootstrap: scratch

%setup
    cp -dR -n --preserve=mode,timestamps ./squashfs-root/usr "${APPTAINER_ROOTFS}/"
    tar xzf ubuntu-bionic-oci-amd64-root.tar.gz -C "${APPTAINER_ROOTFS}"

# Build with Docker.
docker build -t zsync-${zsync2_version}-x86_64 .

# Build with Podman.
podman build -t zsync-${zsync2_version}-x86_64 .

# Build with Singularity.
sudo singularity build "${zsync2_app_image_file%.AppImage}.sif" zsync2.def 

# Buld with Apptainer.
sudo apptainer build "${zsync2_app_image_file%.AppImage}.sif" zsync2.def 

# Run with Docker/Podman.
podman run \
    --mount 'type=bind,source=/,destination=/local_filesystem' \
    --workdir=/local_filesystem/${PWD} \
    zsync-${zsync2_version}-x86_64 \
        zsyncmake2 -u https://test.com/bigfile.bin bigfile.bin

# Run with Singularity/Apptainer.
singularity exec \
    -B ${PWD} \
    ./zsync2-51-4e549b6-x86_64.sif \
        zsyncmake2 -u https://test.com/bigfile.bin bigfile.bin

We have files that are over 100G in some cases, so zsync2 would be nice to have to make sure the download is correct (and can be resumed).

[probonopd]

In an HPC environment, there is in general no option to install AppImage.

...Nor is there a need to install AppImages. Just download, mark as executable, and run with --appimage-extract-and-run if your environment does not provide a working FUSE setup.

[ghuls]

@probonopd I didn't know about that argument, but it doesn't work anyway as HPC clusters have glibc versions from a milion years ago:

❯  ./zsync2-51-4e549b6-x86_64.AppImage --appimage-extract-and-run
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libc.so.6: version `GLIBC_2.22' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libgpg-error.so.0: no version information available (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/usr/bin/../lib/libgcrypt.so.20)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libc.so.6: version `GLIBC_2.25' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/usr/bin/../lib/libcrypto.so.1.1)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libc.so.6: version `GLIBC_2.27' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/usr/bin/../lib/libgssapi_krb5.so.2)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libc.so.6: version `GLIBC_2.25' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/usr/bin/../lib/libgnutls.so.30)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libc.so.6: version `GLIBC_2.22' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/usr/bin/../lib/libgnutls.so.30)
/tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/AppRun: /usr/lib64/libc.so.6: version `GLIBC_2.25' not found (required by /tmp/appimage_extracted_81b6fb70f1742dd32187033f6bf21299/usr/bin/../lib/libcrypt.so.1)

[probonopd]

Interesting! Luckily this will soon be a thing of the past with

• https://github.com/AppImage/AppImageKit/issues/877

[TheAssassin]

What would be needed to provide apptainer packages from this repository? I have zero experience with them (in fact, I just heard of these today for the first time).

[ghuls]

What would be needed to provide apptainer packages from this repository? I have zero experience with them (in fact, I just heard of these today for the first time).

Nowadays providing a docker container with zsync2 should be enough for endusers to build a apptainer image from it.

[TheAssassin]

Well, I'm willing to merge a PR that implements Docker image building with Docker's build-push-action. Would you prepare one, please?