training_policies
training_policies copied to clipboard
HP Borrowing and Scaling
How does HP borrowing work with scaling up a submission?
SWG Notes:
Question: during parameter borrowing, can a submitter re-submit on a larger scale? If not, can a submitter submit a non-converging model on a large system in the hopes of borrowing HPs?
Proposal:
To resubmit benchmark B with a larger scale X:
- You must have submitted some benchmark C at scale Y where Y>=X -- "Prove you can go that big"
- You must have submitted B at a technically comparable so that the only modification in resubmission is HP borrowing
- The scale X must be larger than anything else you submitted for benchmark B
- You cannot withdraw the benchmark C at scale Y and it must be compliant or made compliant
- All submissions must have "converged" (N-1 convergences using olympic scoring)
AI: Review this.
Will revisit for v0.8
@bitfort @petermattson If we are deferring to 0.8, then what did we decide to do for 0.7? I am unclear on the proposed rule or process.