data icon indicating copy to clipboard operation
data copied to clipboard

Add nightly benchmark and trigger in keyword

Open msaroufim opened this issue 2 years ago • 2 comments

This PR runs Nicolas' benchmark suite nightly and stores the results in a github artifact so anyone on the team can inspect them. It's also possible to trigger the benchmark suite manually by typing RUN_BENCHMARK in the PR description which is how this benchmark run was actually triggered

Eventually we'd need to add more benchmarks, log them to the same directory, maybe do a dashboard and of course integrate it into an AWS infra but at least this way the team has a way of easily running benchmarks now without needing to manually spin up machine

h/t @xuzhao9

msaroufim avatar Aug 16 '22 22:08 msaroufim

@NicolasHug is there a setup script I need to run beforehand? https://github.com/pytorch/data/runs/7868534954?check_suite_focus=true#step:9:23

msaroufim avatar Aug 17 '22 00:08 msaroufim

@NicolasHug is there a setup script I need to run beforehand? https://github.com/pytorch/data/runs/7868534954?check_suite_focus=true#step:9:23

That only works on the internal AI cluster. I will message you more details.

NivekT avatar Aug 17 '22 15:08 NivekT