Second mergeable check on PRs

[kohtoa15]

On some PRs, mergeable checks are not correctly updated which leads to the initial check result (created when PR was opened) persisting as a second result, along with the later updated one. Affected PRs cannot be gated, as the first check result blocks merging.

config file:

version: 2
mergeable:
  - when: pull_request.*
    validate:
      - do: title
        must_exclude:
          regex: ^\[WIP\]
      - do: label
        and:
          - must_exclude:
              regex: 'work in progress'
              message: 'work in progress must be excluded'
          - or:
              - and:
                  - must_include:
                      regex: '^SANITY_CHECK$'
                      message: 'SANITY_CHECK must be included'
                  - must_include:
                      regex: '^SANITY_CHECK_PASSED$'
                      message: 'SANITY_CHECK_PASSED must be included'
              - must_exclude:
                  regex: '^SANITY_CHECK$'
                  message: 'SANITY_CHECK must be excluded'

[shine2lay]

@kohtoa15 interesting, from what I know, we can't create two check with the same name, github will just replace the first one with the second. Is this a one off occurrence? Or does it happen everytime?

[kohtoa15]

The problem occurs somewhat rarely, but still once every few weeks. We already reached out to GH support earlier, but they stated that the GH instance has no role in this and that status checks were likely incorrectly updated.

[shine2lay]

@kohtoa15 that's super weird, because from the screen shot, both of the checks are updated correctly to "complete" status (that's why you see either the 'fail' or 'pass' in both of the checks). Also what's weird is that github allows 2 checks with same name, I tried it before as a part of a feature but failed and had to implement feature with Mergeable - check name instead. I am not sure where to even start with debugging this issue