warts
warts copied to clipboard
gh: Allow manual running of the full test suite and erlfuzz
This change introduces two new features:
- Comment "/run-fuzzer" on a pull request to trigger a bot which will run erlfuzz and report the results by replying on the comment thread
- Add the "full-build-and-check" tag to a pull request to force-override CI to always run tests for all applications
The two differ in how they are expressed because I have worked on the basis that:
- Running the fuzzer is expensive, so we probably want to try to limit that to when we know we really need it and explicitly invoke it.
- Running tests for all apps is sometimes needed because the existing mechanism for selecting apps to test is unsound: If you modify stdlib, apps that depend on it won't be analysed, but stdlib itself will. As such, using a tag means that all subsequent test runs for that pull request will return the full signal.
This change also factors out the initial build of OTP in CI into base-build.yaml
so that it can be reused elsewhere, for example, when running erlfuzz.
The mechanism of using comments to execute manual CI jobs could later be extended to add other optional or expensive jobs (e.g. eqWAlizer) so that there is an official means of running them, without requiring that they be part of the usual, automatic CI suite.