Generic provenance: support for container registries

[laurentsimon]

trafficstars

We may want to make it easier for users to generate/upload provenance on a registry. Either we do the upload for them, or we properly document how to do it using cosign themselves.

We can start with the latter (it avoid the needs to handle credentials for upload).

Here's the command that's needed to upload the provenance using cosign: https://github.com/laurentsimon/slsa-github-generator-ko/blob/main/.github/workflows/slsa3-builder.yml#L414-L417

There are already many users who use KEYLESS cosign for their containers. We'd be able to help them have structured SLSA provenance by updating to our generator. Several teams use OPA gatekeeper in k8 to enforce policies, and I think we can provide an OPA policy that is compatible with cosign's

[ianlewis]

This sounds like something we definitely want to do for #57

[laurentsimon]

correct. This issue may be an incremental gain towards the full builder. Since the next release will be the generic provenance; if we can support container signing and verification, we can start migrating users. This will also help us understand the space/features needed for full builder: need to support multiple dockerfiles, etc?

[ianlewis]

I'll look into what a workflow that builds a container would look like and write up some examples. I should put some more time into the docs soon anyway.

[ianlewis]

Removing from milestone per discussion with @laurentsimon

[chipzoller]

I'll also point that Kyverno has built-in and robust support for verification of images (signatures, attestations, and more) right now. For example, here is a sample Kyverno policy which checks to ensure that a SLSA provenance attestation is provided and that it was generated from this very repo. And, as you can see, zero percent of this is code which makes the barrier to entry to software supply chain security very low.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: attest-bom
spec:
  validationFailureAction: enforce
  rules:
    - name: check-slsa-provenance
      match:
        any:
        - resources:
            kinds:
              - Pod
      verifyImages:
      - imageReferences:
        - "ghcr.io/chipzoller/zulu*"
        mutateDigest: true
        attestors:
        - entries:
          - keys:
              publicKeys: |
                -----BEGIN PUBLIC KEY-----
                MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE2kCXNpRME7TtCbzLNyRhdZY5JvmC
                sAeNYYYg5LYpLdYjqtdfly54yEevhBGQUy0vn/tbSOptTnUNcNgnNaBlPg==
                -----END PUBLIC KEY-----
        attestations:
        - predicateType: "https://slsa.dev/provenance/v0.2"
          conditions:
          - all:
            - key: "{{ builder.id }}"
              operator: Equals
              value: https://github.com/slsa-framework/slsa-github-generator/*

You can verify its functionality with a sample Pod using an image I built which provides this attestation:

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: demo
  name: mypod
spec:
  automountServiceAccountToken: false
  containers:
  - name: hello-ko
    image: ghcr.io/chipzoller/zulu:latest

The policy will allow the sample Pod to pass through, and it'll also silently mutate it to use its resolved digest instead of a mutable tag:

$ k get po mypod -o json | k kyverno jp "spec.containers[].image"
[
  "ghcr.io/chipzoller/zulu:latest@sha256:427cd09b93f994224354a53dacc506450e22032618d6b53ecadd6216ab5351fb"
]

[laurentsimon]

Thanks, this is very cool. We were thinking of creating k8's OPA gatekeeper https://github.com/slsa-framework/slsa-verifier/issues/31. (I was not aware of kyverno myself)

I'm curious how you verify the source repo in your policy example? I think you could run the verifier with the -print provenance option and pipe this into the policy engine for additional verification of the content. (The main purpose of the print-provenance option is to allow piping in into policy engines). Can you achieve this with kyverno?

We have plans to expose an API for our verifier, if that's more useful.

Really appreciate all your pointers, it's really useful. We'd really like to collaborate on taking this further, and providing users policy examples with more granular verification (source repo, version tag, or even some of the provenance content we describe in our upcoming blog post https://github.com/slsa-framework/slsa/pull/398).

Let us know if you're interested, we can create an issue on the https://github.com/slsa-framework/slsa-verifier

[chipzoller]

I'm curious how you verify the source repo in your policy example? I think you could run the verifier with the -print provenance option and pipe this into the policy engine for additional verification of the content. (The main purpose of the print-provenance option is to allow piping in into policy engines). Can you achieve this with kyverno?

Kyverno can access the entirety of the attestations by referencing whatever the path is. If you wanted to verify the source repo, you'd just write another simple condition like below. If the field you wanted to check was, for example, under the invocation object it would just be as simple as:

            - key: "{{ invocation.configSource.uri }}"
              operator: Equals
              value: "git+https://github.com/chipzoller/zulu@refs/heads/main"

Definitely glad to write any number of sample policies that would enable users to be able to verify provenance for container images in their clusters via Kyverno. Aside from the documentation, we also have a number posted here with a bunch more due up. Kyverno is a Kubernetes-native policy engine. It's not general purpose like OPA. So if users wanted to verify provenance data for container images being pulled into the cluster, Kyverno is all that's needed. You wouldn't need the SLSA verifier in those cases. Certainly happy to help out here and prove some of these policies out. Just need specifics.

[ianlewis]

@chipzoller Thanks! Kyverno was on my radar but I hadn't played around with it yet. It looks like it has support for verifying cosign's signatures but I wonder if it can search for public keys in rekor, rather then having to provide the public key or cert chain in the policy.

Example policies covering the provenance generated by our builder here would be great! slsa-verifier does a few nice things in addition to verifying the signature, like semver aware version checking. It would be nice to use it as a basis for other verification tools like Kyverno.

[chipzoller]

I wonder if it can search for public keys in rekor, rather then having to provide the public key or cert chain in the policy.

Yes, there is full support for keyless signing.

slsa-verifier does a few nice things in addition to verifying the signature, like semver aware version checking.

Kyverno also has this ability built-in in a couple of ways. The available operators are semver aware and we have even written a custom JMESPath filter which can perform this in an expression. (You'll even notice the example policy used in that documentation example performs a semver comparison on an attestation.)

[laurentsimon]

I wonder if it can search for public keys in rekor, rather then having to provide the public key or cert chain in the policy.

Yes, there is full support for keyless signing.

This is nice. So for us it would be:

- keyless:
    subject: "path/to/re-usableworkflow@v[1-9]+\.[0-9]+\.[0-9]+"
    issuer: ""https://token.actions.githubusercontent.com"

...
- key: "{{ invocation.configSource.uri }}"
              operator: Equals
              value: "git+https://github.com/chipzoller/zulu@refs/heads/main"
...

[chipzoller]

Here's what a completed policy looks like which uses keyless signing to verify both subject and issuer and checks that within the SLSA provenance type (generated from this repo) the URI points to the expected location:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: check-image-keyless
spec:
  validationFailureAction: enforce
  webhookTimeoutSeconds: 30
  rules:
    - name: check-image-keyless
      match:
        any:
        - resources:
            kinds:
              - Pod
      verifyImages:
      - imageReferences:
        - "ghcr.io/chipzoller/zulu:*"
        attestors:
        - entries:
          - keyless:
              subject: "https://github.com/chipzoller/zulu/.github/workflows/slsa-generic-keyless.yaml@refs/heads/main"
              issuer: "https://token.actions.githubusercontent.com"
        attestations:
        - predicateType: https://slsa.dev/provenance/v0.2
          conditions:
          - all:
            - key: "{{ invocation.configSource.uri }}"
              operator: Equals
              value: "git+https://github.com/chipzoller/zulu@refs/heads/main"

Trying with a sample Pod using a container image which satisfies these checks succeeds.

apiVersion: v1
kind: Pod
metadata:
  name: zulu
spec:
  containers:
    - image: ghcr.io/chipzoller/zulu:latest
      name: zulu

$ k apply -f pod.yaml                                                              
pod/zulu created

Create a mismatch in the attestation check and see Kyverno will block the attempt.

$ k apply -f pod.yaml         
Error from server: error when creating "pod.yaml": admission webhook "mutate.kyverno.svc-fail" denied the request: 

resource Pod/default/zulu was blocked due to the following policies

check-image-keyless:
  check-image-keyless: 'failed to verify signature for ghcr.io/chipzoller/zulu:latest:
    .attestors[0].entries[0].keyless: attestation checks failed for ghcr.io/chipzoller/zulu:latest
    and predicate https://slsa.dev/provenance/v0.2'

[laurentsimon]

This is awesome. We'll incorporate this to our doc once we have the generic provenance for dockerfile ready.

[ianlewis]

The generic container workflow supports uploading attestations to registries. I'll close this for now as other features are covered in other issues.