distribution-spec icon indicating copy to clipboard operation
distribution-spec copied to clipboard

Idea: Descriptor as first-class manifest

Open jonjohnsonjr opened this issue 3 years ago • 7 comments

From here.

Often, I just want to upload a blob and download it later. I don't want to use an image manifest or a manifest list as an envelope. There's no config, and I don't want layer semantics. I can't just push the blob, because some registries will GC it if I don't push a manifest that references is. Why can't I just PUT a descriptor?

PUT /v2/.../manifests/v1
Content-Type: application/vnd.oci.descriptor.v1+json

{
  "mediaType": "application/xml",
  "size": 7143,
  "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
  "annotations": {
    "org.opencontainers.description": "cat pics"
  }
}

This would pin the sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e blob, and clients can avoid more complicated envelopes for content.

I can later just GET the thing and fetch the single blob.

cc @awakecoding

jonjohnsonjr avatar Mar 18 '21 22:03 jonjohnsonjr

I really really like this. Pretty much every usage of "OCI Artifacts" I've seen so far has been to upload a single layer blob with some config. It would be sooo much simpler to just be able to wrap that in a descriptor and upload it directly.

dlorenc avatar Mar 28 '21 19:03 dlorenc

I can't actually find any Artifact examples that use more than one layer.

Helm mentions using a second layer for signatures/provenance in their HIP, but I can't find anything else on whether or not it was implemented: https://github.com/helm/community/blob/master/hips/hip-0006.md

Tekton uses a single layer: https://github.com/tektoncd/community/blob/main/teps/0005-tekton-oci-bundles.md

wasm-to-oci uses a single layer: https://github.com/engineerd/wasm-to-oci

Various tutorials all use a single layer:

  • https://docs.microsoft.com/en-us/azure/container-registry/container-registry-oci-artifacts
  • https://aws.amazon.com/blogs/containers/oci-artifact-support-in-amazon-ecr/

dlorenc avatar Mar 28 '21 19:03 dlorenc

i'm for this, but single handedly, the mediaType field will break this idea and be awful to work around

vbatts avatar Nov 03 '21 11:11 vbatts

i'm for this, but single handedly, the mediaType field will break this idea and be awful to work around

Can you elaborate? I'm not sure I follow, but maybe I just haven't had enough coffee this morning.

imjasonh avatar Nov 03 '21 12:11 imjasonh

| Can you elaborate? I'm not sure I follow, but maybe I just haven't had enough coffee this morning.

The mediaType in the oci manifest is the media type of the manifest... mediaType in the descriptor is the media type of that which the descriptor is referring to "the referenced content"... When doing puts/gets there are expectations around the relationship of the content headers and the mediaType. We could add a docType field to the descriptor as a place to specify the media type of the descriptor...

If I remember right the oras effort was looking at using subjectType to specify the referenced content type. overloaded terms ... grr

mikebrow avatar Nov 03 '21 19:11 mikebrow

when we start from a manifest... the config and layers are "descriptors" and thus have a mediaType for the config type document and also one for each layer type. In jon's example the content type header specifies the type of the descriptor.. application/vnd.oci.descriptor.v1+json if we had a ^ docType field we'd have a similar use for docType to match content header with how mediaType is used to match content header for manifest puts/gets...

:-)

course adding a new field will likely result in the sha changing for every descriptor.. can you imagine the hate mail

mikebrow avatar Nov 03 '21 19:11 mikebrow

Do we need a small wrapper?

{
  "mediaType": "application/vnd.oci.descriptor.v1+json",
  "descriptor": {
    "mediaType": "application/xml",
    "size": 7143,
    "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
    "annotations": {
      "org.opencontainers.description": "cat pics"
    }
  }
}

sudo-bmitch avatar Nov 04 '21 00:11 sudo-bmitch