mlcube icon indicating copy to clipboard operation
mlcube copied to clipboard
verified publisher icon mlcommons

TorchBench: Failing unit tests

Open bitfort opened this issue 4 years ago • 2 comments

There are 4 unit tests which are failing.

bitfort avatar Dec 10 '21 17:12 bitfort

Failing Unit Tests:

test_train[pytorch_struct-cpu-jit] test_eval[pytorch_struct-cpu-jit] test_train[pytorch_struct-cpu-eager] test_eval[pytorch_struct-cpu-eager]

Follow up: What is the error?

bitfort avatar Dec 10 '21 17:12 bitfort

Per @davidjurado this is fixed, and there is a PR in the pytorch repo. Next steps to share this with Xu (FB) - PR reviewer.

https://github.com/pytorch/benchmark/pull/559

We still need to:

  1. Make sure GPU tests pass
  2. Compare results when running in MLCube vs. native.

relja128 avatar Dec 17 '21 17:12 relja128