inference icon indicating copy to clipboard operation
inference copied to clipboard
verified publisher icon mlcommons

Update submission checker for v3.1

Open pgmpablo157321 opened this issue 2 years ago • 3 comments

  • [x] Add v3.1 config
  • [x] Add DLRMv2 checks
  • [x] Add GPT-J checks, with additional check for generation length (multiple metric)
  • [x] Update to the last power version
  • [x] Update seeds

pgmpablo157321 avatar Jun 21 '23 20:06 pgmpablo157321

@pgmpablo157321 The update to the latest power version can happen with this PR unless the power WG updates anything else.

arjunsuresh avatar Jun 22 '23 13:06 arjunsuresh

@pgmpablo157321 We need to add GPT-J details here right?

arjunsuresh avatar Jul 11 '23 20:07 arjunsuresh

@arjunsuresh I am not sure, I see there is another conf file here that also doesn’t have the gpt-j details.

@badhri-intel @rnaidu02 Do you know if we need to put any values for gpt-j here?

pgmpablo157321 avatar Jul 12 '23 23:07 pgmpablo157321