Oras 1.1.0 breaks AWS ECR uploads

[Haitham-Shalaby]

What happened in your environment?

Oras 1.1.0 breaks AWS ECR uploads with the following error, so revert back to previous version.

Error: PUT "https://.../manifests/1.21.0-35": response status code 405: unsupported: Invalid parameter at 'ImageManifest' failed to satisfy constraint: 'Invalid JSON syntax'

What did you expect to happen?

No response

How can we reproduce it?

AWS deployment

What is the version of your ORAS CLI?

1.1.0

What is your OS environment?

SLES

Are you willing to submit PRs to fix it?

• [ ] Yes, I am willing to fix it.

[TerryHowe]

I'll take a look at this today

[Haitham-Shalaby]

Thanks

[TerryHowe]

Recreated the problem, but I'm not sure the cause yet.

[qweeah]

@Haitham-Shalaby Can you help clarify what commands are you using?

[TerryHowe]

Confirmed that it did work with 1.0.1

[TerryHowe]

The biggest difference I see is artifactType for 1.0 it was "artifactType":"application/vnd.unknown.config.v1+json" and for 1.1 "artifactType":"application/vnd.unknown.artifact.v1" although I haven't confirmed that is the problem

[qweeah]

The biggest difference I see is artifactType for 1.0 it was "artifactType":"application/vnd.unknown.config.v1+json" and for 1.1 "artifactType":"application/vnd.unknown.artifact.v1" although I haven't confirmed that is the problem

Can you use oras manifest fetch to get the manifest that can be successfully pushed by ORAS 1.0.1? In the manifest, config.mediaType should be set to application/vnd.unknown.config.v1+json.

ORAS 1.1.0 sets artifactType, which violates ECR's implementation of OCI image-spec v1.0.0 and causes the 405 error.

[TerryHowe]

With 1.0

% oras manifest fetch 107907826411.dkr.ecr.us-west-2.amazonaws.com/bundle:latest
{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","config":{"mediaType":"text/yaml","digest":"sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a","size":2},"layers":[{"mediaType":"application/vnd.oci.image.layer.v1.tar","digest":"sha256:0d23844163f3ce4e3a915d613137c47fce32a0ec67a6eb212a64110633e7ced8","size":60635,"annotations":{"org.opencontainers.image.title":"bundle.yaml"}}],"annotations":{"org.opencontainers.image.created":"2023-10-07T14:14:35Z"}}

[TerryHowe]

Pretty print

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "text/yaml",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:0d23844163f3ce4e3a915d613137c47fce32a0ec67a6eb212a64110633e7ced8",
      "size": 60635,
      "annotations": {
        "org.opencontainers.image.title": "bundle.yaml"
      }
    }
  ],
  "annotations": {
    "org.opencontainers.image.created": "2023-10-07T14:14:35Z"
  }
}

[TerryHowe]

That doesn't seem very useful, but when pushing to oci-layout, the diff:

% diff 1*/index.json
6,7c6,7
<       "digest": "sha256:8689e111d40054327f7292e9411b0c7a284e45e97d3aae9ee5355bf42a79f485",
<       "size": 531,
---
>       "digest": "sha256:b4a5164d66e9ba4d0e6f68b2f2af75a2a48c06049c453b43399e925c357c854e",
>       "size": 593,
9c9
<         "org.opencontainers.image.created": "2023-10-07T11:55:07Z",
---
>         "org.opencontainers.image.created": "2023-10-07T12:00:56Z",
12c12
<       "artifactType": "application/vnd.unknown.config.v1+json"
---
>       "artifactType": "application/vnd.unknown.artifact.v1"

[qweeah]

That doesn't seem very useful, but when pushing to oci-layout, the diff:

% diff 1*/index.json
6,7c6,7
<       "digest": "sha256:8689e111d40054327f7292e9411b0c7a284e45e97d3aae9ee5355bf42a79f485",
<       "size": 531,
---
>       "digest": "sha256:b4a5164d66e9ba4d0e6f68b2f2af75a2a48c06049c453b43399e925c357c854e",
>       "size": 593,
9c9
<         "org.opencontainers.image.created": "2023-10-07T11:55:07Z",
---
>         "org.opencontainers.image.created": "2023-10-07T12:00:56Z",
12c12
<       "artifactType": "application/vnd.unknown.config.v1+json"
---
>       "artifactType": "application/vnd.unknown.artifact.v1"

It has nothing to do with the index.json file in oci-layout, the 405 happens because ORAS(1.1.0) generates an image manifest with artifactType set to text/yaml and ECR doesn't allow that.

I guess you are using oras push with --artifact-type set to text/yaml? ORAS 1.0.1 will bake text/yaml into config.mediaType, which is allowed by ECR and that is why 1.0.1 worked.

[jlbutler]

This client defaults to a v1.1 specification, which is not yet released. So to push to ECR (and most other registries), you need to use the --image-spec option with v1.0.

$ oras push --artifact-type application/foo 1234.dkr.ecr.us-west-2.amazonaws.com/oras-test:foo ./foo.txt
Uploading b3f4d7f7146f foo.txt
Uploaded  b3f4d7f7146f foo.txt
Error: PUT "https://1234.dkr.ecr.us-west-2.amazonaws.com/v2/oras-test/manifests/foo": response status code 405: unsupported: Invalid parameter at 'ImageManifest' failed to satisfy constraint: 'Invalid JSON syntax'

$ oras push --image-spec v1.0 --artifact-type application/foo 1234.dkr.ecr.us-west-2.amazonaws.com/oras-test:foo ./foo.txt
Exists    b3f4d7f7146f foo.txt
Pushed [registry] 1234.dkr.ecr.us-west-2.amazonaws.com/oras-test:foo
Digest: sha256:4eb846b56e4b81d1db5c20f0eb285891807f245e13fe15306d502a459c

[qweeah]

Thanks @jlbutler for pointing this out. If you are using oras push, besides from using --image-spec to force generating an image-spec-1.0-compliant artifact, you may also use --config to push an image like below. The generated manifest should also be supported by ECR

$ oras push localhost:6000/spec:v1.0 --config config.json:application/vnd.oci.image.config.v1+json ...
...
Pushed [registry] localhost:6000/spec:v1.0
Digest: sha256:ac75a0db8e295a9205b702ca67cbe61a141b4b3df4d83a900d7daea8f82a30ac
$ oras manifest fetch localhost:6000/spec:v1.0 | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:9c7a54a9a43cca047013b82af109fe963fde787f63f9e016fdc3384500c2823d",
    "size": 1470
  },
...
}

[TerryHowe]

Should probably leave this open for reference

[Haitham-Shalaby]

Thanks all much appreciated inputs

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[TerryHowe]

Keep this open longer

[macropin]

This client defaults to a v1.1 specification, which is not yet released. So to push to ECR (and most other registries), you need to use the --image-spec option with v1.0.

Can this default be changed to v1.0? I would consider it a bug to have defaults that are incompatible with the majority of implementations.

[qweeah]

This client defaults to a v1.1 specification, which is not yet released. So to push to ECR (and most other registries), you need to use the --image-spec option with v1.0.

Can this default be changed to v1.0? I would consider it a bug to have defaults that are incompatible with the majority of implementations.

Changing the default value to v1.0 would make ORAS by default generate artifacts not compliant to guidance in OCI image specification.

[jlbutler]

Changing the default value to v1.0 would make ORAS by default generate artifacts not compliant to guidance in OCI image specification.

Well to be fair, that guidance is from a release candidate spec. The only GA version of either Image or Distribution spec is 1.0.

I think it's the intent of this project to lean into the features specified in the 1.1 spec, and move forward with those features as the default. So I think setting the standard for the default to be 1.1 behavior (and maybe always defaulting to latest in future) does make sense.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[TerryHowe]

I removed the stale label to leave this open a bit longer. Close it in 30 days.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.