velocity icon indicating copy to clipboard operation
velocity copied to clipboard

"Number of pull requests" data appears inaccurate/misleading

Open howardjohn opened this issue 1 year ago • 6 comments

In the velocity reports, we report "The y-axis is the total number of pull requests and issues".

From the query, this is determined by the total amount of PullRequestEvents: https://github.com/cncf/velocity/blob/8e1d1c189b65e2544fae7aec43c6381f9e4b4d82/BigQuery/velocity_cncf.sql#L19C18-L19C34.

A PullRequestEvent does not correlate 1:1 with "a PR" in a way a person would interpret a count of PRs, in my opinion. There are two reasonable approaches (merged PRs or opened PRs, strongly preferring merged PRs), neither of which this counts.

Per docs 'The action that was performed. Can be one of opened, edited, closed, reopened, assigned, unassigned, review_requested, review_request_removed, labeled, unlabeled, and synchronize.'. However, in practice I found this doesn't seem to be the case. Looking at a single day across github:

   1916 reopened
 168129 closed
 193220 opened

Even without the other possible events, we at least appear to be double counting PRs?

howardjohn avatar Nov 02 '23 21:11 howardjohn