Sergii Dymchenko

Actually the commit from https://github.com/pytorch/benchmark/pull/2394 already in main, the PR can probably be just closed. This should be irrelevant to the actual issue.

I looked at ossci-metrics/torchbench-csv/torchao/ on AWS and it doesn't have any recent runs, only couple of runs from Jun 7 and Jun 8. Investigating.

If you look at https://github.com/pytorch/benchmark/actions/runs/10408276046/job/28825370645, WORKFLOW_RUN_ID and WORKFLOW_RUN_ATTEMPT are empty.

I've realized the data is actually in s3, but because WORKFLOW_RUN_ID and WORKFLOW_RUN_ATTEMPT are not populated, that path to the data includes two literal /, so I missed it.

https://github.com/pytorch/benchmark/pull/2426 fixed WORKFLOW_RUN_ID problem. Now https://hud.pytorch.org/benchmark/torchao shows the data. The torchbench part of the run is still in progress, will need to look closely if the data correct later.

> Yeah data doesn't look correct quite yet There was a conflict between my manual run and a nightly run. It was looking differently at some point, with TorchBench, HF,...

Hm, now we have TorchBench and TIMM, but no HF  Investigating further.

OK, I debugged the reason for missing HF. HF is the fastest part (7 hours) vs 17-20 hours for TorchBench and TIMM. Because of that, HF part computed the data...

Direct link to an older date that shows the data for all three: https://hud.pytorch.org/benchmark/torchao?dashboard=torchao&startTime=Fri%2C%2016%20Aug%202024%2023%3A16%3A34%20GMT&stopTime=Fri%2C%2023%20Aug%202024%2023%3A16%3A34%20GMT&granularity=hour&suite=torchbench&mode=inference&dtype=bfloat16&deviceName=cuda%20(a100)&lBranch=main&lCommit=227d4bfece6bb00f0365642ad1d7afd509079ba4&rBranch=main&rCommit=b523f9f9e15b6fb80d10f585d9cf45e0c5e4d10e

> Sounds like the `repo_root` you're passing in is incorrect I want to be able to call my script from a different dir. Like the current dir is `/some/dir` and...