image-spec icon indicating copy to clipboard operation
image-spec copied to clipboard

Add eStargz specification to OCI v1 (support lazy pulling)

Open ktock opened this issue 3 years ago • 8 comments

Resolves #815

This PR adds support of lazy pulling to OCI Image Spec v1 by standardizing eStargz image format as an optional extension. No need to introduce a new layer media type, because eStargz is fully compatible with application/vnd.oci.image.layer.v1.tar+gz.

eStargz has been adopted by tools in the community, including containerd(stargz-snapshotter), Podman, k3s, BuildKit, Kaniko, ko, buildpacks.io, Harbor(acceleration-service). Please refer to https://github.com/containerd/stargz-snapshotter/issues/258 for the complete list.

The following is the summary of this PR:

  • Moves eStargz specification document (estargz.md) from stargz-snapshotter repo to this repo.
  • Adds the link to estargz.md in the existing documents.
  • Adds the annotation org.opencontainers.image.toc.digest defined in estargz.md to specs-go.

ktock avatar Nov 09 '21 11:11 ktock

cc @AkihiroSuda @mattmoor @imjasonh @jonjohnsonjr

ktock avatar Nov 09 '21 11:11 ktock

LGTM on the idea of adding this, but I haven't given this a detailed read.

I think we need to have the maintainers read this one in detail. We may also need a description of how to fetch these in the distribution-spec.

stevvooe avatar Jan 20 '22 18:01 stevvooe

Thank you for the comments.

distribution-spec

Lazy pulling depends on HTTP range request which is already described in the distribution-spec so we don't need changes on the distribution-spec but can add notes that the range request is used by lazy pulling.

ktock avatar Jan 21 '22 05:01 ktock

Needs rebase to make CI green

AkihiroSuda avatar May 29 '22 07:05 AkihiroSuda

Rebased.

ktock avatar Jun 09 '22 10:06 ktock

Thank you for your comments.

there are now multiple (somewhat incompatible) implementations of similar ideas.

Yes, there are several lazy pulling implementations in the community. So it would be great if OCI spec officially describes the format for lazy pulling so that runtimes and builders can adopt it and maintain the implementation while keeping interoperability among them.

ktock avatar Apr 26 '24 07:04 ktock