lotus icon indicating copy to clipboard operation
lotus copied to clipboard

Maintainers have metrics on code review load, makeup, and responsiveness

Open BigLep opened this issue 7 months ago • 7 comments

Done Criteria

Maintainers have public metrics they can look at to see:

  1. PR response time (i.e., the duration between when @lotus-maintainers are notified until when they first reply)
  2. non-draft PR resolution time (i.e., the duration between when a PR is opened as non-draft or marked as "ready-for-review" until when it is merged, closed, or downgraded-to-draft)
  3. non-draft PRs opened vs. merged/closed/downgraded per period (e.g., day)
  4. non-draft PR queue size at a given instant sampled at a constant period (e.g., day)

All of these metrics will have dimensionality on whether the author was a maintainer or not so we can see how we treat contributors outside the team.

Pointers to these metrics should be in the Lotus readme or docs.

Why Important

In 2024Q2, maintainters “felt” they did a better job with PR responsiveness, but let's have the metrics in place so we can “know” if it’s a problem or if we improve or regress.

User/Customer

Maintainers

Notes

  1. We’ll ideally use off-the shelf options than create our own.

BigLep avatar Jul 13 '24 17:07 BigLep