[Whisper] Do we have TEST01 in Whisper?

[wu6u3tw]

Do we have TEST01 in whisper? I do not remember we discussed or promised this before in the TF meeting.

@keithachorn-intel @attafosu

[wu6u3tw]

PR: https://github.com/mlcommons/inference/pull/2277

[hanyunfan]

I’ve emailed the task force chair to confirm this.

[nvzhihanj]

Given the time I would recommend that we waive TEST01 and postpone adding back audit to 6.0

[keithachorn-intel]

Yes, TEST01 is enabled for Whisper.

[keithachorn-intel]

Given the time I would recommend that we waive TEST01 and postpone adding back audit to 6.0

I don't think this is appropriate. TEST01 has been enabled (see https://github.com/mlcommons/inference/tree/master/compliance/nvidia/TEST01/whisper) and is already integrated into the submission checker. If it helps, I can confirm that it performs as expected on our end.

[nvzhihanj]

@keithachorn-intel I was digging through the taskforce meeting notes and don't recall we adding TEST01. But we were able to get past it so don't mind including it. I assume only TEST01 should be needed right?

[taran2210]

Hi, from what I've understood, yes TEST01 was not discussed and perhaps we can decide on that in the post-mortem, but it's normally present for all workloads except LLMs, but if there were not issues with it it can be kept for now, we have also been able to get past it so it should not be an issue

[nvzhihanj]

Sure, @wu6u3tw we can close this issue. We can further investigate next round.