Tracker issue for adoption status

[AkihiroSuda]

(Updated by @ktock and me (@AkihiroSuda))

Summary

• ✅ : Supported
• ➖ : won't be possible
• blank: TBD

Tools	Lazy Pulling of eStargz	Chunk Verification	Creating eStargz	Optimizing eStargz	Handling (pull/push/run) eStargz*
Kubernetes	✅ (w/ containerd, CRI-O)	✅ (w/ containerd)	➖ (doesn't create image)	➖ (doesn't create image)	✅
k3s	✅	✅	➖ (doesn't create image)	➖ (doesn't create image)	✅
kind	✅(needs a customized node image)	✅(needs a customized node image)	➖ (doesn't create image)	➖ (doesn't create image)	✅
containerd	✅	✅	➖ (doesn't create image)	➖ (doesn't create image)	✅
CRI-O	✅		➖ (doesn't create image)	➖ (doesn't create image)	✅
Docker	✅	✅			✅
Podman	✅				✅
nerdctl	✅	✅	✅	✅ (manual optimization)	✅
BuildKit	✅	✅	✅		✅
Kaniko			✅		✅
Buildah					✅
ko			✅		✅
go-containerregistry and crane			✅	✅ (manual optimization)	✅
ctr-remote	✅	✅	✅	✅	✅
buildpacks.io			✅		✅
Harbor			✅		✅ (eStargz can be pushed/pulled as a normal OCI image)

• eStargz is compatible with OCI Image Specification so OCI-compliant tools can handle it in the same way as non-eStargz standard images even if they aren't eStargz-aware.

Kubernetes and distros

Kubernetes

✅ Lazy pulling is possible by using containerd or CRI-O as CRI runtime

k3s

✅ Supported since k3s v1.22. Doc: https://docs.k3s.io/advanced#enabling-lazy-pulling-of-estargz-experimental

kind

Use ghcr.io/containerd/stargz-snapshotter:0.12.1-kind node image e.g.:

$ kind create cluster --name stargz-demo --image ghcr.io/containerd/stargz-snapshotter:0.12.1-kind

• There is a quick start on README: https://github.com/containerd/stargz-snapshotter#quick-start-with-kubernetes

CRI runtimes

containerd

✅ Lazy pulling is supported since containerd 1.4.0

CRI-O

✅ Lazy pulling is supported since v1.22 (which includes https://github.com/cri-o/cri-o/pull/4850)

• limitation: chunk verification is not enabled

High-level container engines

Docker (Moby)

✅ Lazy pulling is supported since https://github.com/moby/moby/commit/5c1d6c957b97321c8577e10ddbffe6e01981617a

Podman

✅ Lazy pulling is supported since v3.3.0 (which includes https://github.com/containers/podman/pull/10214)

• limitation: chunk verification is not enabled

nerdctl

Lazy pulling of eStargz

✅ Supported since nerdctl 0.0.1 https://github.com/containerd/nerdctl/blob/master/docs/stargz.md

Building eStargz images

✅ Supported since nerdctl 0.5.0 https://github.com/containerd/nerdctl/blob/master/docs/stargz.md

• Manual optimization is experimentally supported through --estargz-record-in option.

Image builders

BuildKit

Lazy-pulling base images

✅ Supported since BuildKit 0.8.0 https://github.com/moby/buildkit/blob/master/docs/stargz-estargz.md

Building eStargz images

✅ Supported since BuildKit v0.10.

Usage: buildctl build--output type=image,name=example.com/foo,push=true,compression=estargz,oci-mediatypes=true.

Kaniko

Lazy-pulling base images

❌ Won't be possible

Building eStargz images

✅ Supported since Kaniko 1.4.0 https://github.com/GoogleContainerTools/kaniko/pull/1527

• GGCR_EXPERIMENT_ESTARGZ=1 needs to be specified.

Buildah

Lazy-pulling base images

TBD, will happen after Podman supports lazy-pulling

Building eStargz images

TBD

ko

Building eStargz images

✅ Supported since ko 0.7.0 https://github.com/google/ko/pull/271

• GGCR_EXPERIMENT_ESTARGZ=1 needs to be specified

buildpacks.io

Building eStargz images

✅ Supported since pack 0.16.0, lifecycle 0.10.2

• For pack builder create, build-image need to be pre-converted to eStargz. GGCR_EXPERIMENT_ESTARGZ=1 needs to be specified.
• For pack build, GGCR_EXPERIMENT_ESTARGZ=1 envvar needs to be configured to lifecycle-image. Thus the following image need to be used

  FROM buildpacksio/lifecycle:0.11.3
  ENV GGCR_EXPERIMENT_ESTARGZ=1
  

Registry clients

go-containerregistry and crane CLI

Converting image into eStargz

✅Supported since go-containerregistry 0.3.0 https://github.com/google/go-containerregistry/pull/871

• GGCR_EXPERIMENT_ESTARGZ=1 needs to be specified.
• Manual optimization is supported through --prioritize option.

Registry

Harbor

✅Harbor Acceleration Service webhook enables converting an OCI image into eStargz on registry side.

https://github.com/goharbor/acceleration-service

[ktock]

Thanks! :+1:

[chenk008]

Does Podman need to bump containers/storage version to support stargz?

[ktock]

@chenk008

Does Podman need to bump containers/storage version to support stargz?

Yes. You also need https://github.com/containers/image/pull/1109 and https://github.com/containerd/stargz-snapshotter/pull/301 as well. We'll work on downstreaming these patches to Podman/CRI-O once they are merged.

Please check also the current limitation of lazy pulling feature for these runtimes (https://github.com/containers/storage/pull/795#issuecomment-799057091). Summary:

• ~Podman cannot export (save/push) lazily pulled layer~(fixed in https://github.com/containers/storage/pull/902/commits/2bb8cde363fc9c19b04d3038cc05eb9a8251bc57)
• Podman and CRI-O don't verify lazily-pulled chunks

[AkihiroSuda]

Linking

• https://github.com/rancher-sandbox/rancher-desktop/issues/2707

[gabrieldemarmiesse]

In the new Docker desktop, the containerd image store is in beta. I tried it and it used stargz as the default snapshotter. It has rough edges but pretty cool nonetheless

[giuseppe]

@ktock what is needed for chunk validation in Podman/CRI-O/Buildah?

Isn't that part done by the snapshotter itself?

[harche]

@ktock what is needed for chunk validation in Podman/CRI-O/Buildah?

Isn't that part done by the snapshotter itself?

/cc @AkihiroSuda