inference icon indicating copy to clipboard operation
inference copied to clipboard

Skip scenarios in submission checker

Open yeandy opened this issue 1 year ago • 0 comments

It is sometimes useful to skip a scenario when validating the correctness of the submission package when using submission_checker.py. For example, if we only have Offline results, but not any Server results, we would like the submission checker to still run through all of its validation checks, but not throw an error because either 1) the Server results are not populated in the directory, or 2) the Server results are there, but are INVALID for any reason.

A feature request is to add logic to skip scenario(s).

yeandy avatar May 02 '24 21:05 yeandy