cmd/load: ability to load from oci layout

[Dentrax]

What would you like to be added:

Currently, we can load images from tar files or directly a docker image as follows:

Available Commands:
  docker-image  Loads docker images from host into nodes
  image-archive Loads docker image from archive into nodes

We can also add an oci-layout command here to load an image directly from a folder. As a workaround, we have to do:

$ skopeo copy oci:<OCI_DIR> docker://<DOCKER_IMAGE>
# and then
$ kind load docker-image <DOCKER_IMAGE>

vs.

$ kind load oci-dir <OCI_DIR>

To export your image as an oci-layout, ko have a support for that, for example.

Why is this needed:

To support OCI image layout.

cc @developer-guy @imjasonh

[BenTheElder]

Is there some source that can only produce OCI? ko also supports docker tarballs instead of OCI.

Note that loading is performed in containerd / ctr. kind only streams the tarball and makes some minor optimizations.

It is important that we minimize the dependencies in the kind source code for embedding e.g. in cluster-api, this would probably have to be done as a feature to ctr.

[Dentrax]

Let me ping @estesp @crosbymichael.

Hey! If does it make sense, We'd like to move this issue as a new feature proposal into containerd/ctr. I'm not so familiar with ctr domain, so maybe you can help us point in the right direction.

[estesp]

ctr image import already supports OCIv1 format:

USAGE:
   ctr images import [command options] [flags] <in>

DESCRIPTION:
   Import images from a tar stream.
Implemented formats:
- oci.v1
- docker.v1.1
- docker.v1.2

Is that sufficient for what you are looking for or did I misunderstand what you want to add to ctr?

[Dentrax]

Oh, thanks! Missed that.

In this case, we can easily add a new oci-dir subcommand into kind by adding a new function in nodeutils.go

func LoadImageFromDir(n nodes.Node, imageDir string) error {
 ...
 cmd := n.Command("ctr", "--namespace=k8s.io", "images", "import", imageDir)
 ...
}

As it already describes in import.go.

UX would be:

$ kind load oci-dir <OCI_DIR>

We can get to it if does it make sense. :)

[BenTheElder]

I don't think it will be quite as easy as you say, with image tarballs we are piping in a file not a directory. ctr does not have host filesystem access in this case.

The use case question remains:

Is there some source that can only produce OCI? ko also supports docker tarballs instead of OCI.

[imjasonh]

Yeah, ko's probably a bad example, since it can produce tarballs, and even load directly into KinD all by itself.

I don't know of any tools off the top of my head that can only produce OCI layouts. One nice thing that would enable though is being able to load multiple images in one command, since an OCI layout directory can describe many images.

The typical use case for OCI layouts lately has been as the format for carrying multiple images into a cluster across an airgap. So you could crane pull a bunch of images into an OCI layout, put that on a floppy disk or whatever, load it onto your machine and kind load that. But even that example's a bit moot, since crane pull can also produce regular per-image tarballs, which should work basically the same way.

[BenTheElder]

I don't know of any tools off the top of my head that can only produce OCI layouts. One nice thing that would enable though is being able to load multiple images in one command, since an OCI layout directory can describe many images.

This is also true of a docker archive tarball from docker save etc., kind leverages this if you kind load docker-image one two. It's marginally quicker as it deduplicates layers on the host side.

[BenTheElder]

AIUI OCI-layout is a directory structure, so it's pretty unclear how we'd plumb that through. Existing kind load is roughly implemented by piping to ctr images import. I suppose we could tar and untar it ourselves over a pipe but that seems to be a questionable win over just sticking to archive tarballs ...?

I need to look more at the layout format or seek advice :-)