Incorrect commit id is added for pytest benchmarks

[jonathan-eq]

I am benchmarking with pytest in one repository, and running this workflow from another repository. To get output.json over to the repository using this workflow, I am committing directly to the gh-pages branch. The workflow works very well, but I have noticed that it does not take the commit id from the benchmark output file (which points to which commit the benchmark was ran on), but instead takes the commit id of the commit that added the output.json file in the repository.

[ktrz]

Thank you @jonathan-eq for creating an issue. Are you able to provide some kind of reproduction example or point to a workflow that you are running?

[jonathan-eq]

So I have a closed repository running tests and benchmarks on an onprem github actions runner. Here the output.json file is created by pytest. The final step in that workflow is pushing the file to the gh-pages branch of an open repository. There, this workflow is ran on new commits. We noticed that one of the commits deviated a lot, and when we tried to investigate, we found out that the commit was the one adding the output.json to the open repo, not the one actually being benchmarked. It should be possible to get the commit id from the output.json file.

[ktrz]

Ok @jonathan-eq, I see where the problem is While investigating, I also noticed that pytest output also contains the time the benchmark was run and it's not being used as well. I'll refactor the logic to extract this information from the output if available and only fallback to commit id of the one triggering the workflow and current date if they were not available in the output