pipelines-as-code icon indicating copy to clipboard operation
pipelines-as-code copied to clipboard

failing to report status with GitHub App

Open sm43 opened this issue 3 years ago • 7 comments

with v0.11.0, we received issues where PAC is failing to report status with GitHub App

sm43 avatar Aug 04 '22 13:08 sm43

There are 2 observations after debugging

  • CI has about 8-9 PipelineRuns so creating/updating QPS limit is reached
    • fixed by https://github.com/openshift-pipelines/pipelines-as-code/pull/779 (need to backport)
  • Logs show PAC has added check-run label on PR but actually label is missing from some pipelineRuns

sm43 avatar Aug 04 '22 13:08 sm43

Debugging:

Part of the code from where this is coming https://github.com/openshift-pipelines/pipelines-as-code/blob/85114fb9a994a8d4f3e0592e435dcd1c7d609068/pkg/provider/github/status.go#L143

  • Update call doesn't throw error but label is missing from PipelineRun.
  • Not always reproducible.
  • Tried with 10 Pipelinerun in .tekton, was able to reproduce a few times.
  • also tried replacing Update call with Patch but issue is still there.

sm43 avatar Aug 04 '22 13:08 sm43

Gonna check if this is PAC issue or something in Tekton Pipelines, gonna try to reproduce this directly with TektonPipelines.

sm43 avatar Aug 04 '22 13:08 sm43

also tried replacing Update call with Patch but issue is still there.

https://github.com/openshift-pipelines/pipelines-as-code/pull/788

sm43 avatar Aug 08 '22 08:08 sm43

--- PipelineRun pac-app-pipelines/pac-app-7-fp9f2 v392837 (2022-08-08T13:53:33+05:30) (gen. 1)
    +++ PipelineRun pac-app-pipelines/pac-app-7-fp9f2 v392871 (2022-08-08T13:53:33+05:30) (gen. 1)
    @@ -32,7 +32,6 @@
  build.appstudio.redhat.com/pull_request_number: "68"
  build.appstudio.redhat.com/target_branch: main
  pipelinesascode.tekton.dev/branch: main
  -    pipelinesascode.tekton.dev/check-run-id: "7721544872"
       pipelinesascode.tekton.dev/event-type: pull_request
       pipelinesascode.tekton.dev/git-provider: github
       pipelinesascode.tekton.dev/original-prname: pac-app-7
    @@ -42,9 +41,10 @@
  pipelinesascode.tekton.dev/state: started
  pipelinesascode.tekton.dev/url-org: sm43
  pipelinesascode.tekton.dev/url-repository: pac-app
  +    [tekton.dev/pipeline](http://tekton.dev/pipeline): pac-app-7-fp9f2
    name: pac-app-7-fp9f2
    namespace: pac-app-pipelines
    -  resourceVersion: "392837"
  +  resourceVersion: "392871"
    uid: bcc2a3dc-e9a8-4ba0-9257-2b9308fc2d43
    spec:
      params:

whoever adds tekton.dev/pipeline: pac-app-7-fp9f2 is removing pac label :smile_cat:

sm43 avatar Aug 08 '22 08:08 sm43

in order to avoid a race condition, did a temp fix for now waiting for pipelines label then adding the checkrun id label https://github.com/openshift-pipelines/pipelines-as-code/pull/789

sm43 avatar Aug 09 '22 04:08 sm43

Upstream Tekton Discussion https://tektoncd.slack.com/archives/CLCCEBUMU/p1659949444059689

sm43 avatar Aug 09 '22 04:08 sm43

fixed with https://github.com/tektoncd/pipeline/pull/5597

sm43 avatar Nov 14 '22 11:11 sm43