chains icon indicating copy to clipboard operation
chains copied to clipboard
verified publisher icon tektoncd

[TEP 109] Add feature to extract structured signable targets and store them in subjects and materials in intoto provenance

Open ywluogg opened this issue 3 years ago • 29 comments

Implementation of TEP 109. Example usage for Maven packages:

results:
        - name: img_1-ARTIFACT_INPUTS
           value:
               uri: gcr.io/foo/bar
               digest: sha123@qwe
        - name: mvn1_pkg-ARTIFACT_OUTPUTS
           value:
               uri: maven-test-0.0.1.jar
               digest: sha256@abc
        - name: mvn1_pom-ARTIFACT_OUTPUTS
           value:
               uri: maven-test-0.0.1.pom
               digest: sha256@def
        - name: mvn1_src-ARTIFACT_OUTPUTS
           value:
               uri: maven-test-0.0.1-sources.jar
               digest: sha256@xyz

From this, we will generate subjects in intoto provenence:

{"name": "maven-test-0.0.1.jar", "digest": {"sha256": "abc"}}
{"name": "maven-test-0.0.1.pom", "digest": {"sha256": "def"}}
{"name": "maven-test-0.0.1-sources.jar", "digest": {"sha256": "xyz"}}

In materials:

{"uri": "gcr.io/foo/bar", "digest": {"sha123": "qwe"}}

The feature requires Pipeline v0.38 or later.

ywluogg avatar Jul 15 '22 20:07 ywluogg

/draft

ywluogg avatar Jul 15 '22 20:07 ywluogg

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 75.3% 76.8% 1.5
pkg/chains/formats/intotoite6/intotoite6.go 88.6% 89.7% 1.1

tekton-robot avatar Jul 15 '22 20:07 tekton-robot

/open

ywluogg avatar Jul 18 '22 14:07 ywluogg

/assign wlynch

ywluogg avatar Jul 18 '22 14:07 ywluogg

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 75.3% 76.8% 1.5
pkg/chains/formats/intotoite6/intotoite6.go 88.6% 89.7% 1.1

tekton-robot avatar Jul 18 '22 14:07 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 70.5% 1.6
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 76.7% 2.1

tekton-robot avatar Oct 03 '22 20:10 tekton-robot

/retest

ywluogg avatar Oct 03 '22 23:10 ywluogg

@wlynch Hi Billy, do you mind reviewing this PR? Thanks!

ywluogg avatar Oct 04 '22 14:10 ywluogg

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 70.5% 1.6
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 76.7% 2.1

tekton-robot avatar Oct 04 '22 16:10 tekton-robot

@ywluogg Can you please update https://github.com/tektoncd/chains/blob/main/docs/intoto.md#type-hinting to document the new type hinting fields xx_ARTIFACT_INPUTS and xx_ARTIFACT_OUTPUTS we are introducing here? Thanks

chuangw6 avatar Oct 04 '22 19:10 chuangw6

We might also need to change the function retrieveAllArtifactIdentifiers in https://github.com/tektoncd/chains/blob/main/pkg/chains/storage/grafeas/grafeas.go to get artifacts from StructuredTargets. Also please check if changes are needed for other storage as well.

jagathprakash avatar Oct 05 '22 03:10 jagathprakash

/assign jagathprakash

jagathprakash avatar Oct 05 '22 03:10 jagathprakash

We might also need to change the function retrieveAllArtifactIdentifiers in https://github.com/tektoncd/chains/blob/main/pkg/chains/storage/grafeas/grafeas.go to get artifacts from StructuredTargets. Also please check if changes are needed for other storage as well.

This is a great catch! Thanks @jagathprakash . I just changed that function in grafeas to get URIs directly from intoto subjects instead of calling individual helpers. In future, we don't need to worry the changes made to the Subject function in other places then. So let's keep this out of scope of this PR for now.

chuangw6 avatar Oct 05 '22 13:10 chuangw6

We might also need to change the function retrieveAllArtifactIdentifiers in https://github.com/tektoncd/chains/blob/main/pkg/chains/storage/grafeas/grafeas.go to get artifacts from StructuredTargets. Also please check if changes are needed for other storage as well.

This is a great catch! Thanks @jagathprakash . I just changed that function in grafeas to get URIs directly from intoto subjects instead of calling individual helpers. In future, we don't need to worry the changes made to the Subject function in other places then. So let's keep this out of scope of this PR for now.

Thanks @chuangw6 for fixing this.

jagathprakash avatar Oct 05 '22 16:10 jagathprakash

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 62.5% -6.4
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.3% 0.5
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.0% 0.4

tekton-robot avatar Oct 11 '22 14:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 62.5% -6.4
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.3% 0.5
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.0% 0.4

tekton-robot avatar Oct 11 '22 15:10 tekton-robot

/hold

ywluogg avatar Oct 11 '22 15:10 ywluogg

@ywluogg Can you please update https://github.com/tektoncd/chains/blob/main/docs/intoto.md#type-hinting to document the new type hinting fields xx_ARTIFACT_INPUTS and xx_ARTIFACT_OUTPUTS we are introducing here? Thanks

Will do!

ywluogg avatar Oct 11 '22 15:10 ywluogg

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 62.5% -6.4
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.3% 0.5
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.0% 0.4

tekton-robot avatar Oct 11 '22 15:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 62.5% -6.4
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.3% 0.5
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.0% 0.4

tekton-robot avatar Oct 11 '22 15:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 59.2% -9.7
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.0% 0.3
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.0% 0.4

tekton-robot avatar Oct 11 '22 20:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 59.2% -9.7
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.0% 0.3
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.0% 0.4

tekton-robot avatar Oct 12 '22 00:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 59.2% -9.7
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.0% 0.3
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.0% 0.4

tekton-robot avatar Oct 12 '22 19:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.5% 0.8
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.7% 1.1

tekton-robot avatar Oct 13 '22 15:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 74.1% 5.2
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.5% 0.8
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.7% 1.1

tekton-robot avatar Oct 13 '22 15:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.5% 0.8
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.7% 1.1

tekton-robot avatar Oct 14 '22 18:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 73.8% 4.8
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.5% 0.8
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.7% 1.1

tekton-robot avatar Oct 14 '22 18:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 73.8% 4.8
pkg/chains/formats/intotoite6/extract/extract.go Do not exist 51.4%
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.5% 0.8
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.7% 1.1

tekton-robot avatar Oct 14 '22 19:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 73.8% 4.8
pkg/chains/formats/intotoite6/extract/extract.go Do not exist 51.4%
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.3% 0.5
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.7% 1.1

tekton-robot avatar Oct 17 '22 14:10 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-chains-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/signable.go 68.9% 73.8% 4.8
pkg/chains/formats/intotoite6/extract/extract.go Do not exist 51.4%
pkg/chains/formats/intotoite6/pipelinerun/pipelinerun.go 76.7% 77.3% 0.5
pkg/chains/formats/intotoite6/taskrun/taskrun.go 74.6% 75.4% 0.7

tekton-robot avatar Oct 17 '22 14:10 tekton-robot