Master infra misc improvements

[VladimirSlavik]

[VladimirSlavik]

This is a stub, really. I made these changes while moving towards some other goals.

• I'd like to resolve the situation when two people comment /kickstart... and the tests are started twice, on the very same content. We don't want to run them twice uselessly, period.
• We shouldn't have to run ks tests for a docs change or infra. We could use something like /kickstart-test-vaiwe <free-form text> which would just dump into statuses the green mark and the text after that.

[VladimirSlavik]

I'd like to resolve the situation when two people comment /kickstart... and the tests are started twice, on the very same content. We don't want to run them twice uselessly, period.

It is possible to detect that a job satisfying some criteria is already running, but there is no easy wrapper around that to just cancel the current workflow. There are two paths that almost reach that - concurrency does something similar, but can't cancel the last job (yet?), and skip-duplicate-actions is based on being triggered by a change that has a git commit associated. It would be possible to code this functionality from the ground up, but that's not a small fix any more.

[poncovka]

* We shouldn't have to run ks tests for a docs change or infra. We could use something like `/kickstart-test-vaiwe <free-form text>` which would just dump into statuses the green mark and the text after that.

What about https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/troubleshooting-required-status-checks#handling-skipped-but-required-checks?

[poncovka]

I'd like to resolve the situation when two people comment /kickstart... and the tests are started twice, on the very same content. We don't want to run them twice uselessly, period.

It is possible to detect that a job satisfying some criteria is already running, but there is no easy wrapper around that to just cancel the current workflow. There are two paths that almost reach that - concurrency does something similar, but can't cancel the last job (yet?), and skip-duplicate-actions is based on being triggered by a change that has a git commit associated. It would be possible to code this functionality from the ground up, but that's not a small fix any more.

Based on the documentation, I would expect concurrency to do the job, but maybe I missed something: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-only-cancel-in-progress-jobs-or-runs-for-the-current-workflow

I am be a little worried about skip-duplicate-actions. Sometimes we need to re-run tests on the same code due to infra issues. It looks like this configuration would prevent that.

[VladimirSlavik]

What about https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/troubleshooting-required-status-checks#handling-skipped-but-required-checks?

That's only for checks that are automatically added to the list because of some action associated with the PR. I mean, after they are triggered, they run, and need to reach some resolution. This says just that when the merging rules are considered, any check that's skipped is considered not-failed. But things that run after a comment are not associated with the PR. That's why we must post statuses manually.

I would expect concurrency to do the job

Not implemented. If you declare concurrency, jobs in the same group are queued and only one can run at once. When a new job arrives in the group, it runs if nothing else runs. If something else runs in the group, the new job becomes pending until the running job is finished. Any other pending jobs are by definition older and are all cancelled, except for the new arrival. If you also turn on cancel-in-progress, the still running job is canceled. And that's all it can do. There are at least five RFEs that make sense and this would be one of them.

Sometimes we need to re-run tests on the same code

Concurrency affects only jobs that are a) non-finished and b) in the same group. Jobs for other PRs are in other groups = no effect. Jobs for the same group that already finished or were cancelled manually = no effect. You can run them again.

I do have a bug there - the group is declared by commit hash, while it should be by PR number.

[VladimirSlavik]

Ah, I think I misunderstood the status idea. Sadly, we do post statuses manually, and that does not let us post a "skipped" status: https://docs.github.com/en/rest/commits/statuses#create-a-commit-status

Sometimes, Github actions feels like one big "nope you can't have that".

[VladimirSlavik]

/kickstart-test --testtype smoke