pipeline icon indicating copy to clipboard operation
pipeline copied to clipboard

gitresolver pipes extra resolved data through `status.Annotation`

Open chuangw6 opened this issue 3 years ago • 36 comments

Changes

Change 1:

  • before: The resolvedResource returned by gitresolver may or maynot contain the repo url in status.annotations depending on how users specify the params.
  • now: The resolvedResource returned by gitresolver always contains the repo url. It's the value of param url if it's provided by a user. It's baseURL/<param-org>/<param-repo> if a user only provides a value for org and repo params.

Change 2:

  • before: The revision information in git resolvedResource is the original value provided by user, which can be actual commit hash, or branch name or tag.
  • now: For the branch name and tag, we fetch its actual commit hash and save it in the revision field of the resolvedResource.

Change 3:

  • before: The resolvedResource's annotations are not used/passed to the runobject at all.
  • now: The annotations are passed to the runobject along with the annotations in resolved task.

Change 4: (clean up)

  • before: Some annotation keys (url, commit and path) that can be shared among other resolvers were located in resolution/resolver/git/annotations.go.
  • now: They are moved to resolution/common/annotations.go because those three fields are essentially what SLSA provenance asks for.

Motivation: Tekton Chains needs to capture the source of the config file (pipeline.yaml or task.yaml) in the SLSA provenance's predicate.invocation.configSource field.

  • Always passing all 3 fields from resolver to runobject will make it possible for Chains to capture the information without modifying user-defined spec. (change 1 + 3)
  • Aligning revision field to be actual hash helps Chains capture the actual diest which is required for the predicate.invocation.configSource field. (change 2)

Current status of this PR (evolving)

  • This change has been done for Task level for now to gather some thoughts and feedback. This PR needs to be updated to do for Pipeline level too.

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • [ ] Has Docs included if any changes are user facing
  • [ ] Has Tests included if any functionality added or changed
  • [x] Follows the commit message standard
  • [ ] Meets the Tekton contributor standards (including functionality, content, code)
  • [x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
  • [x] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
  • [x] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

Pass the remote refs that resolver received from ResolutionRequest to TaskRun/PipelineRun status.annotation so that Chains can capture the data and record in the provenance.

chuangw6 avatar Aug 30 '22 23:08 chuangw6

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

tekton-robot avatar Aug 30 '22 23:08 tekton-robot

/test all

chuangw6 avatar Aug 31 '22 00:08 chuangw6

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remote/oci/resolver.go 72.3% 71.4% -0.9

tekton-robot avatar Aug 31 '22 00:08 tekton-robot

/test tekton-pipeline-unit-tests

chuangw6 avatar Aug 31 '22 00:08 chuangw6

/test all

chuangw6 avatar Aug 31 '22 01:08 chuangw6

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/resources/taskref.go 85.2% 86.0% 0.8
pkg/remote/oci/resolver.go 72.3% 71.4% -0.9

tekton-robot avatar Aug 31 '22 01:08 tekton-robot

/retest

chuangw6 avatar Aug 31 '22 01:08 chuangw6

/retest

abayer avatar Aug 31 '22 13:08 abayer

/assign

abayer avatar Aug 31 '22 13:08 abayer

Oh, and this should be done for Pipelines, not just Tasks. I'd do all of that in one PR.

abayer avatar Aug 31 '22 14:08 abayer

Oh, and this should be done for Pipelines, not just Tasks. I'd do all of that in one PR.

Yeah, that's what I planned to do in this PR. I just did for Tasks for now to gather some thoughts first. I've updated the description to reflect that. Thanks @abayer!

chuangw6 avatar Aug 31 '22 14:08 chuangw6

/test all

chuangw6 avatar Aug 31 '22 15:08 chuangw6

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/resources/taskref.go 85.2% 86.0% 0.8
pkg/remote/oci/resolver.go 72.3% 71.4% -0.9

tekton-robot avatar Aug 31 '22 15:08 tekton-robot

/kind feature

chuangw6 avatar Sep 08 '22 14:09 chuangw6

FYI, the git resolver undergoes some significant changes in #5450.

abayer avatar Sep 09 '22 14:09 abayer

FYI, the git resolver undergoes some significant changes in #5450.

Thanks for the heads up @abayer

chuangw6 avatar Sep 09 '22 19:09 chuangw6

/lgtm

alhuizenga avatar Sep 12 '22 14:09 alhuizenga

@alhuizenga: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot avatar Sep 12 '22 14:09 tekton-robot

Also, really nicely written PR @chuangw6, even I could understand it!

alhuizenga avatar Sep 12 '22 14:09 alhuizenga

Also, really nicely written PR @chuangw6, even I could understand it!

Thanks @alhuizenga !! Appreciate it!

chuangw6 avatar Sep 12 '22 14:09 chuangw6

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/pipelinerun.go 84.9% 84.9% -0.0
pkg/reconciler/pipelinerun/resources/pipelineref.go 89.7% 90.0% 0.3
pkg/reconciler/taskrun/resources/taskref.go 85.7% 86.0% 0.3
pkg/reconciler/taskrun/taskrun.go 80.7% 80.9% 0.2
pkg/remote/oci/resolver.go 72.3% 71.4% -0.9
pkg/resolution/resolver/git/resolver.go 84.6% 84.4% -0.2

tekton-robot avatar Sep 13 '22 19:09 tekton-robot

The following is the coverage report on the affected files. Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/pipelinerun.go 84.9% 84.9% -0.0
pkg/reconciler/pipelinerun/resources/pipelineref.go 89.7% 90.0% 0.3
pkg/reconciler/taskrun/resources/taskref.go 85.7% 86.0% 0.3
pkg/reconciler/taskrun/taskrun.go 80.7% 80.9% 0.2
pkg/remote/oci/resolver.go 72.3% 71.4% -0.9
pkg/resolution/resolver/git/resolver.go 84.6% 84.4% -0.2

tekton-robot avatar Sep 13 '22 20:09 tekton-robot

@abayer Please take a look since this is closely related to https://github.com/tektoncd/pipeline/pull/5450. Thanks.

Not sure change 1 is the correct way to get the real url if users only provide a value for repo and org. In github example, it would be https://api.github.com/<org>/<repo>.git Thoughts?

chuangw6 avatar Sep 13 '22 20:09 chuangw6

/retest

chuangw6 avatar Sep 13 '22 20:09 chuangw6

/retest

chuangw6 avatar Sep 13 '22 20:09 chuangw6

/approve

I'd change the title to reflect that this isn't git resolver specific, since it should be doing this for all resolvers now, right?

abayer avatar Sep 19 '22 13:09 abayer

/approve

I'd change the title to reflect that this isn't git resolver specific, since it should be doing this for all resolvers now, right?

Thanks for reviewing this PR @abayer ! Correct, this change will be applied to other resolvers. This PR for now only implements the gitresolvers's part. I'll open other PRs to implement that for oci and catalog resolvers.

chuangw6 avatar Sep 19 '22 14:09 chuangw6

Correct, this change will be applied to other resolvers. This PR for now only implements the gitresolvers's part. I'll open other PRs to implement that for oci and catalog resolvers.

@chuangw6 would you mind opening an issue to track this?

dibyom avatar Sep 19 '22 18:09 dibyom

Also while we are at it, can we update the gitresolver docs to mention the additional status.annotations that it will always provide?

dibyom avatar Sep 19 '22 18:09 dibyom

@chuangw6 would you mind opening an issue to track this?

Sure! Created here https://github.com/tektoncd/pipeline/issues/5522

chuangw6 avatar Sep 19 '22 19:09 chuangw6