Config option/knob for mksquashfs compression settings

[zeronewb]

Version of Apptainer

apptainer version 1.1.5-1.fc37

Expected behavior

Presense of config and cli options for mksquashfs. For example: apptianer.conf mksquashfs comp zstd runtime apptainer build foo.si foo.def -comp lz4/zstd etc.

Actual behavior

Right now apptainer use default mksquashfs flags for compression, or GzipFlag if defined.

Steps to reproduce this behavior

Build any sif image from def file.

What OS/distro are you running

NAME="Fedora Linux"
VERSION="37 (Workstation Edition)"
ID=fedora
VERSION_ID=37
VERSION_CODENAME=""
PLATFORM_ID="platform:f37"
PRETTY_NAME="Fedora Linux 37 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:37"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f37/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=37
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=37
SUPPORT_END=2023-11-14
VARIANT="Workstation Edition"
VARIANT_ID=workstation

How did you install Apptainer

RPM from fedora repo.

[GodloveD]

It probably would not be difficult to instantiate a cli interface or env vars to control the compression type through mksquashfs. But I wonder if this might make the resulting container more difficult or potentially impossible to use. In other words, what if the container creator used a non-standard type of compression when they created the container. Would the end user need to know what type of compression was used and add some flag to the Apptainer call to run the container?

[frejanordsiek]

I just tested what would happen if Apptainer 1.1.16 is given a SIF that has zstd compression. In short, it will not run it. It instead gives the error while checking system partition header: error while checking squashfs header: corrupted image: unknown compression algorithm value 6. This means that SIF images compressed with other algorithms are not backwards compatible.

My recipe to make such a SIF file is given below. Note that I am using sudo at two of the steps to make sure that ownership and permissions inside the squashfs don't change when it is extracted and to make the new squashfs image regardless of the permissions inside the old image.

apptainer build test_default.sif docker://docker.io/library/alpine:3.17
sudo unsquashfs -o 36864 -d files test_default.sif
cd files
sudo mksquashfs * ../new_image.sqfs -comp zstd
cd ..
head -c36864 test_default.sif > test_zstd.sif
cat new_image.sqfs >> test_zstd.sif

[nathanweeks]

The unknown compression algorithm value 6 error also occurs when mounting a SquashFS image file that was compressed using the mksquashfs -comp zstd option.

[DrDaveD]

I can imagine making apptainer flexible enough to recognize other compression algorithms but I'm not sure it's a good idea to make it easy for people to generate non-backward compatible SIF files.

[GodloveD]

Maybe @zeronewb can provide some info about what would be gained by allowing this flexibility? If so, please re-open.

[nathanweeks]

I can imagine making apptainer flexible enough to recognize other compression algorithms but I'm not sure it's a good idea to make it easy for people to generate non-backward compatible SIF files.

I was thinking more so of having the ability to mount a file system from a squashfs file (e.g., containing data) that was compressed with one of the other compressors that squashfs-tools supports (mksquashfs 4.6 lists 5 supported compressors). zstd in particular seems to offer both a better compression ratio and performance (especially decompression) than gzip. edit: This is probably a different use case than the one described in the original issue