benchmark icon indicating copy to clipboard operation
benchmark copied to clipboard

Update score and plot to compute subscore for gpu/cpu/train/eval

Open nikithamalgifb opened this issue 4 years ago • 0 comments

The current torchbenchmark framework reports one single benchmark score and doesn’t give a way to calculate the subscore for GPU/CPU/train/eval/eager/JIT. So we need a way to get the benchmark subscore by calculating the subscore for each of the following 8 configurations:

* train-cuda-eager
* train-cpu-eager
* eval-cuda-eager
* eval-cpu-eager
* train-cuda-jit
* train-cpu-jit
* eval-cuda-jit
* eval-cpu-jit

Here are the two subtasks required to accomplish this:

  • [ ] Update the subscore depending on the configuration
  • [ ] Generate 9 plots . One for each of the above configuration(8) and then one with the whole score

nikithamalgifb avatar Jan 28 '21 17:01 nikithamalgifb