Decision tasks associated with a `github-push` event displayed in the Github Checks UI of a pull request

[ahal]

Describe the bug Decision tasks associated with a github-push event can be displayed in the Github Checks UI of a pull request.

To Reproduce

Steps to reproduce the behavior:

1. Create a feature branch my-feature
2. Push it to a branch of the same name on the main repo (i.e, not a fork)
3. Open a pull request from that branch in the Github UI
4. Make a new commit in the feature branch and push

This will generate both a github-push event since we are pushing to a branch on the main repo, as well as a github-pull-request event, since we are updating a pull request, therefore spawning two Decision tasks. So far I believe this is expected behaviour.

The bug is that taskcluster-github will sometimes display the result of the github-push created Decision task in the Github checks UI rather than the github-pull-request one. I believe whichever Decision task runs last will be the one that gets shown (though this is unconfirmed). In other words, it's a race to see which one is the one we display to the user. This is a problem because the differing tasks_for value means that the two graphs could have significant differences (i.e, github-push might be level 3, and github-pull-request might be level 1).

Expected behavior Only Decision tasks created via github-pull-request events should be visible in checks interface.

Taskcluster version 43.2.0

[ahal]

For context this came up in the mozilla-vpn-client repo where this Decision task from a github-push was displayed in the checks UI of this PR.

Note: I've worked around this particular instance in the .taskcluster.yml file, but the general problem still exists.

[djmitche]

I think that this might be an issue with how GitHub indexes checks. I believe it's by (owner, repo, sha, check-name), but that would need to be checked. The fix here would be to make those keys different, perhaps by including the tasks_for value in the name. I think that's why tools like Circle have something like "circle-ci/push" and "circle-ci/pr"?

[petemoore]

This seems like https://bugzilla.mozilla.org/show_bug.cgi?id=1588436

[ahal]

This is also not limited to Decision tasks, here's a case where it happens on a build task: https://bugzilla.mozilla.org/show_bug.cgi?id=1761841

[ahal]

I discovered it's possible to work around this in the .taskcluster.yml by adding tasks_for to the Decision task name, e.g like in this PR: https://github.com/mozilla-mobile/mozilla-vpn-client/pull/3835/files

The individual tasks will still overwrite one another, but at least we can find them via their respective Decision tasks. This is just a work around though and I still think having separate check suites (one per tasks_for) is the best way forward.