Support for filtering architectures on multi-arch copy

[jlamande]

This is a feature request.

When copying an image from docker.io, I need to capture many architectures as I have multiple runtime target platforms but supporting only 2 architectures : linux/armv8 and linux/amd64. So I use the all value for --multi-arch as system wouldn't fill my needs. Thanks for this great feature 👍🏻

Unfortunately when copying, for example nginx from docker.io, with the --multi-arch all mode, skopeo retrieves all the available architectures. If I am right, this will result in a waste of resources (storage) on my target registry with unneeded images (5 of 7 for the current nginx).

Having a way to filter the architectures to retrieve would be greatly appreciated 🙂

[mtrmac]

Thanks for your report.

Yes, the design of --multi-arch anticipates something like that, but it’s not yet available.

In the immediate term, it might be possible to script around this by manually collecting the relevant set of images (by manually parsing the index, compare #1283 ), then copying them one by one, and finally using --multi-arch=index-only. That would work only if the destination registry supports sparse multi-arch images, though.

[Jamstah]

As an FYI, I'm looking to get registry support in here: https://github.com/distribution/distribution/issues/3628

I'd be interested to know more about your use case and the software you're running.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[jbg]

In our case we're mirroring public images into our private ECR repositories so that we can pull them from systems with no Internet access. We need linux/amd64 and linux/arm64, but many manifests also contain armv7 and even Windows images. In many cases the Windows images are multiple GiB in size whereas the Linux images are a few tens of MiB, so the impact of mirroring the unneeded Windows images is relatively large. It would be great to be able to filter these out.

[Jamstah]

It's currently possible with skopeo by copying the specific architecture images and the index separately using --multi-arch=index-only from this PR: https://github.com/containers/skopeo/pull/1511

However, you'd have to try it and see if ECR will accept the index if all the platform images are not in the repository. If it doesn't, you'd need to raise a request with AWS to ask them for that capability.

[astraldawn]

Identical use case, with Artifactory as destination registry.

Have not tried --multi-arch=index-only. Index with images not in repository directly uploaded to Artifactory via API.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[zerda]

Here's a little tip, you can use docker buildx imagetools to push multi-arch image.

Test with github.com/docker/buildx v0.10.3 79e156beb11f697f06ac67fa1fb958e4762c0fab.

# create and push multi-arch image, predownload source image is not required

# destination: example.com/library/eclipse-temurin:17-jdk-focal
# source: eclipse-temurin@sha256:c9f82f7... (linux/amd64)
#         eclipse-temurin@sha256:42d35f0... (linux/arm64)
$ docker buildx imagetools create -t example.com/library/eclipse-temurin:17-jdk-focal \
    eclipse-temurin@sha256:c9f82f747ab63080ca88e5c863525669411454dcb3dae371b9ae9ca11fb80746 \
    eclipse-temurin@sha256:42d35f09787b300a4d111bb253ddb1cdac1bb918dff227a201adbb405e5bf624

And with inspect command, it's really easier to get image name with digest.

$ docker buildx imagetools inspect docker.io/library/eclipse-temurin:17-jdk-focal
Name:      docker.io/library/eclipse-temurin:17-jdk-focal
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:5c0cdf9b7e5fe51e9498dd76d6119f14f251b1ae29589fdc97396276887cfc10

Manifests:
  Name:      docker.io/library/eclipse-temurin:17-jdk-focal@sha256:c9f82f747ab63080ca88e5c863525669411454dcb3dae371b9ae9ca11fb80746
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      docker.io/library/eclipse-temurin:17-jdk-focal@sha256:1115cdd6d6ca5ca90c2e76c16cdbbee37e610e52a1e2a11637eb596be4fcdc9a
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7

  Name:      docker.io/library/eclipse-temurin:17-jdk-focal@sha256:42d35f09787b300a4d111bb253ddb1cdac1bb918dff227a201adbb405e5bf624
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64/v8

  Name:      docker.io/library/eclipse-temurin:17-jdk-focal@sha256:929ac7b3b9b5cfab083af4fd2106bf98b82098d5025f3d3ea610ea8d7f87589a
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/ppc64le

  Name:      docker.io/library/eclipse-temurin:17-jdk-focal@sha256:83e559131d46af10a1e559a0f05b3e079f146eaee543a919acbfb9e9b8f307ba
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/s390x

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rdjones517]

Just wanted that this feature would be very useful for my project as well.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.