ci(github): workflow to ensure PR description & commit message parity
Description
As a maintanier/reviewer I want to have automation that makes sure that people don't just fill out the PR description on GH and leave their commit message basically empty so that the real explanation for the changes are contained in the git history and not in a database that we don't control.
Acceptance Criteria
- Implement the solution via a GitHub check like the PR title linting that we have in place right now.
- Make sure to have a manual override built-in (available only to the reviewers on a case-by-case basis) so that in special cases the check can be marked as passing (but the default is to fail the check and not let the PR get merged without the commit message being legible)
- Set it up in a way so that it's part of the checks executed for every pull request and it gets executed when the PR gets updated as well (e.g. if I force push something and change the commit message to something non-legible it should start failing again)
Further context: You can use this GitHub Action from the marketplace to make API calls directly from the workflow YAML: https://github.com/octokit/request-action
The above then enables you to retrieve the PR description and the PR commit's message through the GitHub API.
Finally with this information being available you can run a comparison and issue a warning state on a check you wrote so that people who are the authors of a pull request where the commit message and the pull request description are mismatched can get to know about this problem.
To determine the exact endpoints to use to retrieve the information above, check the GitHub REST API docs: https://docs.github.com/en/rest?apiVersion=2022-11-28
So to tie it all together: You'd create a check job in one of our yaml files which would then run on pull requests and perform the data retrieval through the GitHub API to get the PR description and commit message, compare the two and set the status of the check to failed or succeeded based on that.
@zondervancalvez Some test to demonstrate what it should do on a high level:
Test case 1:
PR Description: "asdf" Commit 1 message in the PR: "asdf"
^^ The check should pass here because everything from the PR description is present.
Test case 2: PR Description: "asdf - some other information as well" Commit 1 message in the PR: "asdf"
^^ The check should FAIL here because the person updated the PR description with new information that is missing from the git commit message.
Test case 3:
PR Description: "asdf" Commit 1 message in the PR: "abcdefg" Commit 2 message in the PR: "asdf"
^^ The check should PASS here because there was a commit in the pull request that contained the information from the PR description. It's OK if commit 1 didn't have it because at least commit 2 did so the information won't be lost.