Speed up CI by splitting checks

[onbjerg]

Describe the feature

Currently we run all checks three times:

• Once per PR
• Once per merge group
• Once when merged on the main branch

Technically, running all checks in the merge group and on the main branch is redundant, as the checks carry over (and are logically similar as the merge group rebases). We could disable checks here that are already run in the merge group.

Some checks are also run on every PR that could just be run in the merge group - if they fail, the PR is removed from the merge group. I think only a few checks qualify here, probably the sync tests and the benchmark-related checks.

So the plan would be:

1. Prevent checks that are run in the merge group from being run on main
2. Move some checks off of PRs and only run them in the merge groups (more expensive tests/checks). If the check fails in the merge group, the PR is moved back and there are links to figure out why it was removed

Looking for thoughts on this - the primary con would be that you could technically break e.g. benchmark compilation in a PR but only know it once it is being merged, but the obvious pro is faster CI.

Additional context

No response

[mattsse]

Move some checks off of PRs and only run them in the merge groups (more expensive tests/checks).

this sounds reasonable, especially the bench job

Prevent checks that are run in the merge group from being run on main

this doesn't really hurt, right? because is triggered after merge

[onbjerg]

Not really, but I think there is a maximum number of jobs that can be run per repository at one time, so reducing this reduced run minutes and should (I think) speed up how fast pending jobs are picked up by runners