training_policies
training_policies copied to clipboard
Ensure that all submitters have a review
We discussed this in the past (https://github.com/mlperf/training_policies/issues/250) but I'm not clear we ever actually came up with a process for assigning reviewers. We should discuss what this will be.
SWG:
There are going to be a lot of submissions this time. Every submission deserves a review. There should be a process to do this.
For this round, we will follow this algorithm:
- Stack rank by number of submissions
- Assign reviewers in pairs walking down the stack rank
- If an odd number of reviewers, the bottom 3 in the stack rank will review each other.
This is to be done on Friday!
SWG:
Algorithm subject to review committee modification to ensure resources are well suited for review.