Produce signed .NET container images

[mthalman]

Signing a container image involves using a digital signature that includes information about the image, such as its hash value, and is created using a private key. The consumer can then verify this signature using a public key from a trusted source.

This provides the following benefits:

1. Authenticity: Ensures that the image comes from a trusted source and has not been tampered with.
2. Integrity: Guarantees that the image has not been modified since it was signed.
3. Trust: Builds trust between the image publisher and the consumers of the image.

We should produce .NET container images that are signed to provide these additional benefits.

Validation would be supported via the notation CLI from the Notary project. The entire workflow of signing and validation would be the same as is described in Announcing Image Signing for Windows Containers

[mthalman]

Here's the link for how to configure the infrastructure to produce signed images: https://eng.ms/docs/more/containers-secure-supply-chain/signing (internal link).

[mthalman]

[Triage]

We should determine whether image signing will be done across all images (6.0, 7.0, 8.0) or on a version-by-version basis. Would we apply image signing to already released versions or would this be something only applied to an upcoming version (i.e. 8.0)?

[richlander]

One option is:

• Start signing with .NET 8 only.
• Prove that out and ensure no issues.
• Wait for folks to ask for signing with older versions.

[mthalman]

Related work item: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/2060356 (internal link)

[lbussell]

[Triage] This will use the ORAS tool to sign the images. An issue should be logged to add the tool to the ImageBuilder Dockerfile. This is also needed for https://github.com/dotnet/docker-tools/issues/1201.

The signing process doesn't have to happen at build time, so we don't need a solution for using the ORAS tool on Windows agents. Including it in the Linux ImageBuilder image will be sufficient.

There are some other open questions about the implementation process that should be investigated:

1. Although the signing doesn't have to happen at build time, would it be beneficial to do this at build time? This could allow us to test our image signature in the test stage.
2. Will there be any performance hit to the build for the signing process?

[lbussell]

A couple of key points to highlight:

• Images themselves are not signed. What is signed is a JSON payload containing info about an image digest (which refers to a manifest). Since a digest necessarily changes whenever the content of an image is different, signing the digest of the image is sufficient to attest that we created the image.
• The spec for the signature payload is HERE.

Here's a summary of the mechanics for signing images:

1. Build images as normal
2. For each digest to be signed, create a payload according the the Notary V2 spec:

   {
     "targetArtifact": {
       "mediaType": "application/vnd.oci.image.manifest.v1+json",
       "digest": "sha256:<...>",
       "size": <length of the digest's manifest text>
     }
   }
   

3. Send the payload to our signing service to be signed. Applicable keys for signatures and attestations are listed in Signing container images (eng.ms).
   • This can be accomplished with an Azure DevOps task. There is dnceng-specific documentation for this in "Converting-AzDo-EsrpCodeSigning-Yml-Task..." (dev.azure.com). This could make our builds look very messy if we need to run a separate pipeline step for each image that was built. If there's no way to sign multiple artifacts with one pipeline step, then I would consider that a point in favor of making image signing a separate pipeline stage.
   • The alternative is to use "ESRP.exe" which appears to be Windows-specific - not ideal for our case.
   • There is also an in-development Az CLI extension that could be used, but it's not well documented and that option needs more investigation.
4. Use ORAS to attach and push the signature to the ACR:

     oras attach myregistry.azurecr.io/namespace/namespace/hello-world:v0.0.1 \
       --artifact-type 'application/vnd.cncf.notary.signature' \
       ./payload.json:application/cose \
       -a "io.cncf.notary.x509chain.thumbprint#S256=[\"aaaaaaaaaa\", \"bbbbbbbbbb\"]
   

   The thumbprint is the list of SHA-256 thumbprints of the signing certificate and certificate chain (including root) that were used for signature generation.

The next step is to run an experiment with signing some of our images in our internal/testing ACR.

[lbussell]

Another question is "Which digests should we sign?".

We could either sign only individual digests, or sign all digests and digest lists that we push.

I lean towards signing both digests and digest lists, since it would give a better user experience when checking if multi-platform images are signed. For example, if we didn't sign digest lists, you could get an output like this:

PS> oras discover mcr.microsoft.com/dotnet/nightly/runtime-deps:8.0
Discovered 0 artifact referencing 8.0

[richlander]

Would be good if we provided a mini tutorial on how to use the tool for our images. Would also be good to show to how to validate the contents. That would be a step harder but I think worthwhile.

[lbussell]

Would be good if we provided a mini tutorial on how to use the tool for our images. Would also be good to show to how to validate the contents. That would be a step harder but I think worthwhile.

@richlander, I completely agree, I wouldn't consider this feature complete without documentation showing how to verify the signature of an image.

[mthalman]

Ideally that documentation would be hosted on the MAR portal since it is not specific to .NET images. Perhaps the portal should also have some indication of which images are signed. /cc @terencet-dev

[terencet-dev]

Thanks for the tag and feedback. Support for referrers (such as signatures) is still part of our backlog at the moment. /cc @nhu1997