Self upgrade PRs don't run checks

[SgtCoDFish]

See e.g. https://github.com/cert-manager/website/pull/1545

The required check didn't run until I closed + reopened the PR. This PR should instead be automatically merged if tests pass. From a discussion with @inteon and @wallrj this is apparently a known issue.

At a minimum we should document this flaw. Ideally it should be fixed of course!