REQ conditioned on context values

[azrael417]

Hi all,

I am wondering if it is possible to create checks which are conditional on context values. For example for the HPC deepcam benchmark, the optimizers which can be chosen are LAMB and Adam(W). LAMB has more parameters than Adam so I require those to be logged and checked but only if opt_name == LAMB. Is there a way to achieve this with the current EXACTLY_ONE, AT_LEAST_ONE and AT_LEAST_ONE_OR? It would be sufficient to have a requirement checking which marks it optional depending on the context dict.

Best Thorsten

[shangw-nvidia]

Yeah it's already supported. Below is an example for resnet, depending on which optimizer is used, different rules are checked against: https://github.com/mlcommons/logging/blob/master/mlperf_logging/compliance_checker/1.0.0/closed_resnet.yaml#L11 https://github.com/mlcommons/logging/blob/master/mlperf_logging/compliance_checker/1.0.0/closed_resnet_lars.yaml https://github.com/mlcommons/logging/blob/master/mlperf_logging/compliance_checker/1.0.0/closed_resnet_sgd.yaml

You could also add a "BEGIN" block to store which optimizer is used ahead of time, then adjust the rules for each key accordingly; example: https://github.com/mlcommons/logging/pull/107/files#diff-f80dfc858917b7fb2d6f66ee98b242b88112531a509a667e677594ebf74c2377R1