training_policies icon indicating copy to clipboard operation
training_policies copied to clipboard

Ensure that all submitters have a review

Open jonathan-cohen-nvidia opened this issue 4 years ago • 2 comments

We discussed this in the past (https://github.com/mlperf/training_policies/issues/250) but I'm not clear we ever actually came up with a process for assigning reviewers. We should discuss what this will be.

jonathan-cohen-nvidia avatar Jun 25 '20 15:06 jonathan-cohen-nvidia

SWG:

There are going to be a lot of submissions this time. Every submission deserves a review. There should be a process to do this.

For this round, we will follow this algorithm:

  • Stack rank by number of submissions
  • Assign reviewers in pairs walking down the stack rank
  • If an odd number of reviewers, the bottom 3 in the stack rank will review each other.

This is to be done on Friday!

bitfort avatar Jun 25 '20 16:06 bitfort

SWG:

Algorithm subject to review committee modification to ensure resources are well suited for review.

bitfort avatar Jun 25 '20 16:06 bitfort