[docs] Verifying provenance with Kyverno

[ianlewis]

Docs on how to verify provenance generated by the generic workflow with Kyverno.

Kyverno docs on verifying images is here: https://kyverno.io/docs/writing-policies/verify-images/

[ianlewis]

cosign attest sets the intoto statement's subject to the "repo" name which I assume is the image path (e.g. ghcr.io/ianlewis/myimage) and the repo sha256 digest. We should be able to reproduce how cosign attest works and allow it to work with Kyverno. https://github.com/sigstore/cosign/blob/a2e1c1d1f3f1835d70e1138ca4ac6156215d20dd/pkg/cosign/attestation/attestation.go#L171-L179

Will test more early next week.

[chipzoller]

I'd be glad to provide assistance/review here.

[ianlewis]

I tested out Kyverno with the generic workflow and it currently doesn't work because of how we store the certificate use to sign the attestations.

Kyverno seems to call VerifyImageAttestations from the cosign API here.

I have the following cluster policy set:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: check-slsa-attestations
spec:
  validationFailureAction: enforce
  webhookTimeoutSeconds: 30
  rules:
    - name: check-all-keyless
      match:
        any:
          - resources:
              kinds:
                - Pod
      verifyImages:
        - imageReferences:
            # Replace with your image. Wildcard values are supported.
            - "ghcr.io/ianlewis/actions-test:*"
          attestors:
            - entries:
                - keyless:
                    subject: "https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@refs/tags/v1.1.1"
                    issuer: "https://token.actions.githubusercontent.com"
          attestations:
            - predicateType: https://slsa.dev/provenance/v0.2
              conditions:
                - all:
                    - key: "{{ invocation.configSource.uri }}"
                      operator: Equals
                      value: "git+https://github.com/ianlewis/actions-test@refs/tags/v0.0.11"
                    - key: "{{ invocation.configSource.entryPoint }}"
                      operator: Equals
                      value: ".github/workflows/generic-container.yaml"
                    - key: "{{ regex_match('^https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@refs/tags/v[0-9].[0-9].[0-9]$','{{ builder.id}}') }}"
                      operator: Equals
                      value: true

When I try to create the following Pod:

apiVersion: v1
kind: Pod
metadata:
  name: actions-test
spec:
  containers:
    - name: actions-test
      image: ghcr.io/ianlewis/actions-test:v0.0.11
  restartPolicy: Never

I get the following error:

$ kubectl apply -f actions-test-pod.yaml 
Error from server: error when creating "actions-test-pod.yaml": admission webhook "mutate.kyverno.svc-fail" denied the request: 

resource Pod/default/actions-test was blocked due to the following policies

check-slsa-attestations:
  check-all-keyless: |-
    failed to verify signature for ghcr.io/ianlewis/actions-test:v0.0.11: .attestors[0].entries[0].keyless: no matching attestations:
    no certificate found on attestation

I'm not sure yet why it's failing but I suspect it has to do with how we are storing the cert w/ Rekor.

It's the same error you get locally if you try to verify using keyless:

$ COSIGN_EXPERIMENTAL=1 cosign verify-attestation ghcr.io/ianlewis/actions-test:v0.0.11
Error: no matching attestations:
no certificate found on attestation
main.go:52: error during command execution: no matching attestations:
no certificate found on attestation

[ianlewis]

A really simple way to get around this might be to have the generic generator just spit out the predicate and then run cosign attest --type slsaprovenance --predicate predicate.json $IMAGE to upload the provenance.

We can only do that if we know it's a container though.

[chipzoller]

And this is when you generate the in-toto attestation file and attach it with cosign attach attestation? If so, yes, it's the same issue. I guess I'm not clear though where your code comes into play if all you're doing is pumping out the attestation file and it's up to cosign to attach it.

[laurentsimon]

I'd be glad to provide assistance/review here.

+1. Would it beneficial to add a section directly in https://kyverno.io/docs/writing-policies/verify-images/? We could simplify our doc in this repo and link to the official doc, in order to avoid duplication (and also, your doc looks so much better than ours :)). Let us know what you think.

[chipzoller]

Would it beneficial to add a section directly in https://kyverno.io/docs/writing-policies/verify-images/?

Sure, we could do that. I think it would make sense to add that process once the "official" container image provenance generator is released and tested, also probably needs cosign fixes or "enablements" per the issue I created.

[laurentsimon]

A really simple way to get around this might be to have the generic generator just spit out the predicate and then run cosign attest --type slsaprovenance --predicate predicate.json $IMAGE to upload the provenance.

any reason we would not use this command/API? (It has a different format from the one we use for binaries, but that's not a problem since we need to know we're signing a container image vs an arbitrary artifact anyway, right?)

We could have a generic base64-oci-subject, which would be a list of OCI names?

[laurentsimon]

Would it beneficial to add a section directly in https://kyverno.io/docs/writing-policies/verify-images/?

Sure, we could do that. I think it would make sense to add that process once the "official" container image provenance generator is released and tested, also probably needs cosign fixes or "enablements" per the issue I created.

can you paste the link to the issue?

[chipzoller]

Oh right, you're not Ian :) Issue reference: https://github.com/sigstore/cosign/issues/2027

[laurentsimon]

Fyi https://nirmata.com/2022/07/14/securing_base_images/. May be interesting to build on top of this post to showcase our SLSA generators / builders

[chipzoller]

We could probably do that!

[laurentsimon]

Other interesting links: https://boxboat.com/2021/12/06/secure-supply-chains-kyverno/ https://kyverno.io/docs/writing-policies/verify-images/

[chipzoller]

I think in order to properly showcase this, though, we would need some resolution on that linked Cosign issue. We (everyone, really) needs to be able to fetch all the attestations to consider in policy decisions without prior knowledge of how they may have been supplied to an image (attest vs attach).

[ianlewis]

@chipzoller I think for now our workflow can use cosign attest to sign and upload the provenance. We created a separate workflow to be container aware for that reason. The new workflow works with Kyverno and others that expect it provenance to be uploaded this way.

I will push a bit more on the issue you linked since I think it's necessary for consistency in cosign but I don't think it's a blocker for the container workflow.

[chipzoller]

Agreed, and even if there's no resolution on that thread, as long as you're using cosign attest and passing the un-attested predicate, there should be no issues.