moby
testing: try to build every commit in a PR
When multi-commit PR is submitted, every commit should build successfully(just build is fine, no need to run tests for every commit). We should verify this in the CI to make sure this is actually the case. Otherwise, it is hard to be sure about that in big PRs like #3075 .
For PR's like https://github.com/moby/buildkit/pull/3075, it should probably also do a vendor-check for each commit. Ideally we'd also have a way to re-trigger CI and it doing a merge of the target branch into the PR branch before running.
I think it's possible by iterating github.base_ref to github.head_ref and create a matrix of each commit from there.
it should probably also do a vendor-check for each commit
If it easy to detect the commits that change the vendor then maybe. But the main reason for doing this is to make sure the git bisect works correctly(in addition to checking that commits are correctly separated) so the validations don't matter much for that case.
Yep you can do this in GHA - have a job (e.g. the base one) generate some JSON and use the fromJson() function in your matrix def on the later job
If we do, we should probably add a sensible limit to that list; if a PR has a bad rebase, pulling in 1000 commits (yes, it happened), we don't want to start a queue of 1000 runs 😁
Wonder if in complement of #3624 we could also enable the Merge Queue feature https://github.blog/changelog/2023-02-08-pull-request-merge-queue-public-beta/