medperf
medperf copied to clipboard
[FEATURE] Make --model and --evaluation mlcubes really optional during `medperf test run`
If you are a benchmark owner, you have to implement three cubes: data_prep, model and evaluation. The issue is that you cannot run (or implement) them one-by-one: if the benchmark is not submitted still, you have to pass all three mlcubes before able to run any of them.
I propose to make --model and --evaluation cubes totally optional during medperf test run so Benchmark Owner can start implementing just from data-prep cube, not worrying about other parts, thus develop cubes one by one.