logging icon indicating copy to clipboard operation
logging copied to clipboard

Validate number of logging entries

Open mwawrzos opened this issue 3 years ago • 1 comments

The MLPerf v7.0 logging was described by a MLPerf Training Logging Spec v0.7 document, which includes a definition of the number of expected logs.

The committee decided to drop support for this document, and the checkers should be the only source of logging rules. With this decision, we lost the definition of required logs, and submitters don't know, what is expected.

I propose to update verifiers to add support for log entries count validation. This test should be a part of the tool, that has knowledge about a number of nodes/accelerators in the submission. The package checker may be the right choice.

mwawrzos avatar Mar 22 '21 12:03 mwawrzos

@mwawrzos could you clarify if we still need to address this for v1.1?

xyhuang avatar Jul 27 '21 17:07 xyhuang