training_policies icon indicating copy to clipboard operation
training_policies copied to clipboard

[HPC] Proposal: Improve results website

Open nvaprodromou opened this issue 2 years ago • 3 comments

Introduction:

After collecting feedback from engineers, clients, and press, NVIDIA presented a list of proposals that aim to improve the popularity of the MLPerf HPC benchmark suite. Please see our slide deck for more information on our feedback gathering process and insights.

Proposal: Improve (modernize) results website

Slide 12 in proposals slide deck.

This proposal aims to improve the popularity of the MLPerf HPC benchmark suite by improving on the following aspects:

  1. Simplifies results parsing and understanding [Improves press interest]

Discussion

Pros:

  1. Quick, easy fix
  2. Reduces publics/reporters’ confusion when faced with our results

Cons:

  1. Some reporters have already invested time to understand our current terminology.
    • We believe that reporters will appreciate the clarity and simplicity of the revised terminology.

nvaprodromou avatar Feb 07 '23 17:02 nvaprodromou

I don't know if this really is a quick easy fix, unless you have a simple, specific suggestion.

I believe MLCommons broadly recognizes the need for improved results display. We can rally for this with David, and in meetings, but not sure what else we can do.

sparticlesteve avatar Feb 27 '23 16:02 sparticlesteve

Hi @nvaprodromou I concur that the results presentation is problematic. I'm actually working with a product manager and we are interviewing folks about how they are used with an eye towards redesigning the results.

This is an area we want to invest in, but it isn't quick :)

TheKanter avatar Mar 03 '23 03:03 TheKanter

I agree it's not quick and easy - sorry for that. May be the result of copy/paste between filing PRs. Or maybe I grossly misjudged the effort needed. I forget why that statement is there.

nvaprodromou avatar Mar 06 '23 18:03 nvaprodromou