Improve `prePR` and CI feedback, esp. for new contributors
After an extensive discussion on Discord about "What prePR means to you", I think we roughly came to this consensus:
prePRshould only do polish tasks, which are essentially things that can be fixed by the task itself. Additionally, thepolishcheck should be a separate job in CI, like suggested in https://github.com/typelevel/sbt-typelevel/issues/93.- We should try and relax the CI-requires-approval-for-new-contributors, since it really kills the feedback loop. I'm pretty sure there's a mode to only require it for new GH users, and I bet an admin can set this for the whole org.
Roughly, prePR/polish tasks should meet this criteria:
- it happens without any configuration or control
- it's infallible when it runs
- once you run it locally, the check in ci will never fail
- things that check something and fix it if needed on their own
As much as CI failures are annoying, at least maintainers can help. We don't want prePR to discourage contributors from creating a PR in the first place by failing with e.g. mima issues that they don't understand or know how to fix.
There's also a lot of interest in an idea to ask CI to run prePR for you like in https://github.com/typelevel/sbt-typelevel/issues/69, but it's possibly tricky to implement.
See also https://github.com/typelevel/sbt-typelevel/pull/203#issuecomment-1063234479.
Note that some scalafix are "polish" (e.g., organize imports), and some require intervention (e.g., no fs2.Compiler[Sync]). And some of those that are "polish" are semantic rules, and therefore relatively slower.