tektoncd
Add support for specifying artifacts to be processed without outputting all URIs to results
Feature request
I would like it to be possible to have Chains process output artifacts (i.e. signing, and provenance generation) without having to output the image URL and digests directly to Tekton results.
While it is possible to use the IMAGES result to process multiple output artifacts from a TaskRun, the space available for storing information in results is limited. While there are options for increasing the space available, these options are not possible to be configured on a task-by-task basis (xref: https://github.com/tektoncd/pipeline/issues/8448).
Since the signing support is already present for artifacts in a container registry (i.e. it is assumed that there is a configured container registry), it would be beneficial if it would be supported to produce a specific artifact which can have all of the digest-pinned image references for signing. This can be as simple as pushing a json file to a container registry (i.e. oras push) and recording the URI/digest of that image manifest in the TaskRun result. Chains would then pull the artifact and process the list of identifiers.
Use case
- As a task writer, I would like to be able to generate many artifacts and images to be signed by Chains without requiring that the task is run in a cluster configured with larger results sizes.
A variation of this might be the solution to a use case I have just come across.
I have a Task that produces at least one file which is stored in an OCI registry as a blob/layer. It may produce many files, each stored as an individual blob. I wrap those blobs in an Image Manifest for two reasons: 1) to avoid garbage collection of "dangling" blobs by the registry; and 2) to group multiple files in a single result to avoid the result size limitation you mention.
I want the digest of those files (blobs) to be added to the list of subjects in the SLSA Provenance. I don't really care about the digest of the Image Manifest, but I don't see the harm in adding that to the list as well. Currently, there is no way to achieve this. If I emit the digest to the blob, Chains fails because it assumes that the reference is to an Image Manifest.
I can imagine a config option, e.g. enable-artifact-expansion, that when enabled, it would cause Chains to check if the Image Manifest has a non-empty value for artifactType. If so, it would iterate through the layers of the manifest, and add each reference to the list of subjects. It may also sign each one of those (but we should keep #1346 in mind).
@arewm, would that work for your use case as well? Or are you thinking more along the lines of an Image Index-like functionality?
Hm, I think that it is similar. It proposed a different implementation detail. Both approaches, along with another that just came to mind can effectively be used to expand the "type hinting" for Chains from simple Tekton results to OCI artifacts.
Original suggestion
Put a list of URIs/digests in a file
- Benefits: This allows you to specify files in separate repositories; an OCI artifact is used which can be referenced by digest
- Drawbacks: It imposes additional validation requirements to Chains; it allows images to be specified in separate repositories
Variation 1
Produce a dedicated Image manifest with references to specific blob layers
- Benefits: Validation is maintained by the OCI registry; higher confidence that necessary secrets are present; all subjects are known to exist in the same OCI repository; an OCI artifact is used which can be referenced by digest
- Drawbacks: If we want to add multiple Image Manifests it isn't intuitive that each would be its own layer
I don't really care about the digest of the Image Manifest, but I don't see the harm in adding that to the list as well.
You may actually want to add this to the list of subjects. If you want to be able to reuse the blob as a cache, you will need to be able to reference that cache again. A tag on an Image manifest may be helpful for this and its provenance may then be beneficial.
Variation 2
Produce a dedicated Image index with references to specific subjects
- Benefits: This would support the referencing of multiple Image manifests and Image indices; this could be compatible with variation 1 if you wanted to further expand references in a linked Image manifest; all subjects would be contained in the same repository; an OCI artifact is used which can be referenced by digest
- Drawbacks: Blobs would still need to be wrapped in an image manifest to be referenced