Not supporting in-toto DSS payloadType in verify-attestation

[zregvart]

Description

I've created an in-toto attestation and uploaded it to a image registry. When I tried to use cosign verify-attestation --policy my.cue ... the policy written in CUE was not asserted. When debugging it seems to have been ignored because of the payloadType check[1] against known policies, and application/vnd.in-toto+json not being defined/handled there.

As far as I understand the application/vnd.in-toto+json should be the correct DSSE payload type for the in-toto attestation and this should be allowed to assert against a supplied policy.

This is the image reference of the image and attestation:

$ docker pull zregvart/blank:latest # image
$ docker pull zregvart/blank:sha256-76addd9802ebcdb49bd3beac67d1b6c4ff2514bf867bb02f2b3605f633ba7f3b.att # attestation

Given this CUE policy:

predicate: passed: false

Verify attestation should fail, but it doesn't

$ cosign verify-attestation docker.io/zregvart/blank:latest \
  --key cosign.pub \
  --policy policy-test.cue
Verification for docker.io/zregvart/blank:latest --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
{"payloadType":"application/vnd.in-toto+json","payload":"eyJfdHlwZSI6Imh0dHBzOi8vaW4tdG90by5pby9TdGF0ZW1lbnQvdjAuMSIsInByZWRpY2F0ZVR5cGUiOiJjb3NpZ24uc2lnc3RvcmUuZGV2L2F0dGVzdGF0aW9uL3YxIiwic3ViamVjdCI6W3sibmFtZSI6ImluZGV4LmRvY2tlci5pby96cmVndmFydC9ibGFuayIsImRpZ2VzdCI6eyJzaGEyNTYiOiI3NmFkZGQ5ODAyZWJjZGI0OWJkM2JlYWM2N2QxYjZjNGZmMjUxNGJmODY3YmIwMmYyYjM2MDVmNjMzYmE3ZjNiIn19XSwicHJlZGljYXRlIjp7IkRhdGEiOiJ7XG4gIFwiX3R5cGVcIjogXCJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YwLjFcIixcbiAgXCJwcmVkaWNhdGVUeXBlXCI6IFwiaHR0cHM6Ly9leGFtcGxlLmNvbS9UZXN0UmVzdWx0L3YxXCIsXG4gIFwic3ViamVjdFwiOiBbXG4gICAge1xuICAgICAgXCJuYW1lXCI6IFwiZG9ja2VyLmlvL3pyZWd2YXJ0L2JsYW5rXCIsXG4gICAgICBcImRpZ2VzdFwiOiB7XG4gICAgICAgIFwic2hhMjU2XCI6IFwiNzZhZGRkOTgwMmViY2RiNDliZDNiZWFjNjdkMWI2YzRmZjI1MTRiZjg2N2JiMDJmMmIzNjA1ZjYzM2JhN2YzYlwiXG4gICAgICB9XG4gICAgfVxuICBdLFxuICBcInByZWRpY2F0ZVwiOiB7XG4gICAgXCJwYXNzZWRcIjogdHJ1ZVxuICB9XG59XG4iLCJUaW1lc3RhbXAiOiIyMDIyLTAzLTI5VDEyOjA0OjA1WiJ9fQ==","signatures":[{"keyid":"","sig":"MEUCIQDIJIRqkvxWyZWczeFwfBYRh2PffJHVQmq+IRszfKqo4QIgLRRSWZ9zghJ6l45OMcdqYYt41snTwe2eBGKmxoCLvl8="}]}

The attestation was created using cosign:

$ cosign attest --key cosign.key --predicate predicate.json docker.io/zregvart/blank:latest

[1] https://github.com/sigstore/cosign/blob/ba50ee05c87701f532c63ec6d0fd0d7f3c03d34d/cmd/cosign/cli/verify/verify_attestation.go#L204

[vaikas]

zomg :) I just hit this myself and went to create an issue and found this :)

[vaikas]

Looks like there's a few folks that have been hitting this issue, and I personally don't have enough confidence / knowledge on what all kinds of other formats might break here, so in addition to removing the sanity check that does not seem to me to work, I also started sketching what e2e tests may look like (lots of refactorings, etc. clearly needed to keep on adding, but hopefully a start to see where I was thinking this may go, and then should be able to run e2e tests locally as well) so that we can keep adding tests here to validate the payload formats as well as predicate types, etc. https://github.com/sigstore/cosign/pull/1685

Also would be nice to get some of the rego tests fixed here into e2e tests: https://github.com/sigstore/cosign/pull/1672

I personally kind of like having these kinds tests that use cosign to both create / verify the attestation so we get nice round-trip tests.

Thoughts?

You can seen an example passing run here (includes one passing cue policy and one failing): https://github.com/sigstore/cosign/runs/5747763172?check_suite_focus=true

[zregvart]

Thoughts?

Very much so, test-first approach should be the way to go here. I think #1680 is a stronger candidate to fix this than what I patched together to make it work for my use case I'll close #1680 in favor of #1685.

[zregvart]

For my use case I wanted custom in-toto statements i.e. with custom predicateType and predicate like:

{
  "_type": "https://in-toto.io/Statement/v0.1",
  "predicateType": "https://example.com/TestResult/v1",
  "subject": [
    {
      "name": "docker.io/zregvart/blank",
      "digest": {
        "sha256": "76addd9802ebcdb49bd3beac67d1b6c4ff2514bf867bb02f2b3605f633ba7f3b"
      }
    }
  ],
  "predicate": {
    "passed": true
  }
}

I missed that by default cosign attest uses custom type, so the predicate JSON above would be wrapped, i.e.:

{
  "_type": "https://in-toto.io/Statement/v0.1",
  "predicateType": "cosign.sigstore.dev/attestation/v1",
  "subject": [
    {
      "name": "index.docker.io/zregvart/blank",
      "digest": {
        "sha256": "76addd9802ebcdb49bd3beac67d1b6c4ff2514bf867bb02f2b3605f633ba7f3b"
      }
    }
  ],
  "predicate": {
    "Data": "{\n  \"_type\": \"https://in-toto.io/Statement/v0.1\",\n  \"predicateType\": \"https://example.com/TestResult/v1\",\n  \"subject\": [\n    {\n      \"name\": \"docker.io/zregvart/blank\",\n      \"digest\": {\n        \"sha256\": \"76addd9802ebcdb49bd3beac67d1b6c4ff2514bf867bb02f2b3605f633ba7f3b\"\n      }\n    }\n  ],\n  \"predicate\": {\n    \"passed\": true\n  }\n}\n",
    "Timestamp": "2022-03-29T12:04:05Z"
  }
}

And the policy above, would match. I've now used the URI https://example.com/TestResult/v1 as predicate type and the CUE policy fails as expected with predicate: passed: false and passes with predicate: passed: true.

This now works with #1685:

$ cosign attest --key cosign.key --predicate predicate.json --type https://example.com/TestResult/v1 docker.io/zregvart/blank:latest
$ cosign verify-attestation --key cosign.pub --policy policy-test.cue docker.io/zregvart/blank:latest

Hope that this might help as a test case for the end to end tests.

[vaikas]

@zregvart and @lcarva thanks much. My plan is to:

• Convert scaffolding to be used as an action so it's easier to grok what's happening. Also hoping that it will then be able to export all the env variables, etc.
• Refactor the actual tests to be in test/attestation_tests.sh or something so that they are easier to grok.
• I'd like to take a peek at magefiles, if y'all are experts here, mayhaps that is something that we could look at using for e2e tests.

Thoughts about this plan of attack?

[zregvart]

@vaikas looks like you have a clear idea of what needs to be done, my only concern is that some of it seems orthogonal to this issue (e.g. the use of Mage), though I do see how that would make the code easier to grok and adding/refactoring tests always gets a big +1 from me.