inference icon indicating copy to clipboard operation
inference copied to clipboard

Remove backwards compatibility for the submission checker. Update sub…

Open arjunsuresh opened this issue 1 year ago • 1 comments

…mission_checker.py

It makes sense to remove the backwards compatibility of the submission checker in the master branch. We can always use a specific release of the inference repository to get the submission checker for that round of submission.

arjunsuresh avatar Apr 11 '24 09:04 arjunsuresh

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

github-actions[bot] avatar Apr 11 '24 09:04 github-actions[bot]

Thanks Arjun, I think we are okay with the change as long as it doesn't break the behavior of 4.0 and 4.1 existing benchmarks. Have you tested the workloads?

nvzhihanj avatar May 14 '24 20:05 nvzhihanj

Thank you @nvzhihanj for reviewing. Yes, it is tested for 4.0 submission repo by the github action.

arjunsuresh avatar May 14 '24 21:05 arjunsuresh

@mrmhodak NVIDIA members experienced with submission checker also agreed that we can drop the backwards compatibility and thus simplify the submission checker a lot. Arjun's PR is a good idea and we approve.

nv-ananjappa avatar May 15 '24 21:05 nv-ananjappa