Add --multi-arch=sparse:…

[achilleas-k]

This is partially related to https://github.com/containers/image/issues/1874.

Our use case (Image Builder / osbuild) requires transferring a container and its manifests (and particularly the manifest list) using a portable format that doesn't affect the image itself. We also don't want to be downloading all the images for all the architectures just to get the manifest list. In the end, it should be possible to run the container by referencing its manifest list digest without needing access to the original registry (or even an internet connection).

@mtrmac suggested using --multi-arch=index-only to store the manifest list in a dir: and then getting the image separately, which means eventually, these would need to be merged. I've used the following script to accomplish this, but I'd like confirmation that this is the right way.

Using Fedora 37 on Quay.io as an example

skopeo copy --preserve-digests docker://quay.io/fedora/fedora@sha256:176454f0e89d7bda8b8b577bfd855f5cb3854234d781855baef82cb057b0529e dir:container-archive/image

skopeo copy --preserve-digests --multi-arch=index-only docker://quay.io/fedora/fedora@sha256:176454f0e89d7bda8b8b577bfd855f5cb3854234d781855baef82cb057b0529e dir:container-archive/index
cp -a container-archive/image container-archive/merged

digest=$(sha256sum container-archive/image/manifest.json | cut -f1 -d' ')

mv container-archive/merged/manifest.json "container-archive/merged/${digest}.manifest.json"

cp container-archive/index/manifest.json container-archive/merged/manifest.json

To verify that it worked:

sudo skopeo copy --preserve-digests dir:container-archive/merged 'containers-storage:[overlay@/var/lib/containers/storage+/run/containers/storage]quay.io/fedora/fedora@sha256:176454f0e89d7bda8b8b577bfd855f5cb3854234d781855baef82cb057b0529e'
sudo podman run --rm --pull=never quay.io/fedora/fedora@sha256:176454f0e89d7bda8b8b577bfd855f5cb3854234d781855baef82cb057b0529e uname -a

runs the container without needing to pull anything.

Given the current dir: format, do you see any issues with this way of merging?

Is this something that can be added to the tooling (particularly, skopeo, either as part of the copy command or separately as a "merge" command?

[mtrmac]

Yes, that script looks good, with the current format of the dir: directories.

---

I don’t see that a “merge” operation of two dir: directories is very well-defined in general; dir: directories are typically expected to contain all components, the --index-only case being the major exception.

I think we’d want something like skopeo copy --multi-arch=sparse:[amd64,arm64] or possibly skopeo copy --sparse=[system], to preserve the original manifest list, but to only copy some of the images. That half-exists via copy.CopySpecificImages + copy.Options.Instances (where the caller can specify instances by digests, but not by platform).

A counterpart would be something like skopeo copy --multi-arch=filter:[amd64,arm64], needed by quite a few, e.g. https://github.com/containers/image/pull/1707 (but not preserving the manifest list digest)

---

(To correctly prioritize this in the business context, please file this in Jira.)

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[snail2sky]

Yes, that script looks good, with the current format of the dir: directories.是的，该脚本看起来不错，具有 dir: 目录的当前格式。

I don’t see that a “merge” operation of two dir: directories is very well-defined in general; dir: directories are typically expected to contain all components, the --index-only case being the major exception.我不认为两个 dir: 目录的“合并”操作总体上定义得非常明确； dir：目录通常应包含所有组件，--index-only 情况是主要例外。

I think we’d want something like skopeo copy --multi-arch=sparse:[amd64,arm64] or possibly skopeo copy --sparse=[system], to preserve the original manifest list, but to only copy some of the images. That half-exists via copy.CopySpecificImages + copy.Options.Instances (where the caller can specify instances by digests, but not by platform).我认为我们需要像 skopeo copy --multi-arch=sparse:[amd64,arm64] 或可能的 skopeo copy --sparse=[system] 之类的东西，以保留原始清单列表，但只复制一些图像。这一半通过 copy.CopySpecificImages + copy.Options.Instances 存在（调用者可以通过摘要指定实例，但不能通过平台指定实例）。

A counterpart would be something like skopeo copy --multi-arch=filter:[amd64,arm64], needed by quite a few, e.g. containers/image#1707 (but not preserving the manifest list digest)对应的东西类似于 skopeo copy --multi-arch=filter:[amd64,arm64]，很多人都需要，例如Containers/image#1707（但不保留清单列表摘要）

(To correctly prioritize this in the business context, please file this in Jira.)（要在业务环境中正确确定此问题的优先级，请在 Jira 中归档。）What a great idea, such a shame，This function has not been completed