opam-repo-ci
opam-repo-ci copied to clipboard
Improve reliability of revdeps checks
When new package releases encounter revdeps errors, it can be really time consuming (and probably pretty unreliable) to try to audit the failures to see if the build/test failures have any relationship to the new release. We can use this issue to gather ideas on how to make the reported results more reliable and easier to navigate.
Two ideas to try to improve the relevance of revdeps errors:
- Add heuristics to filter out irrelevant build failures (e.g., those resulting from missing sources cannot be related to updates of dependencies)
- Account for diff between revdeps failures accepted during pervious version release, and new ones encountered on the current release (see https://github.com/ocaml/opam-repository/pull/26816#issuecomment-2454701201).
Both of these could benefit from reviewer input during review time, allowing reviewers to mark revdeps failures as irrelevant, e.g., with the addition of a regex on certain errors, or by marking problem packages.
Another improvement that is already planned is to not report errors for packages that fail due to an intermediate dependency failure: e.g., if we have a dependency chain like A <- B <- C and B fails, we do not also want a report on the inevitable failure of C due to B.