dorny
New generated reports appear under an old run
Describe the bug
When I first added this action, and run the workflow manually (run_number 300 - screenshot1) everything worked fine and the report was generated on the same run summary page.
Screenshot1:
But when I run the same workflow again multiple times WITHOUT COMMITING NEW CHANGES to the repo (run_number 301), the action runs successfuly but the generated reports doesn't appear on the corresponding run summary page.
Screenshot2:
After checking, I found out that all newly generated reports are being added to the first run page (number 300) as you can see in the following screenshot.
Screenshot3:
But again, if I push new commits to the repo with very minimal change (adding a blank line), and run the workflow, the issue is resolved for that specific run and the generated report appears where it's supposed to:
To Reproduce
Steps to reproduce the behavior:
- Run the workflows manually and wait until the report is generated.
- Without commiting any new changes to the repo, run the workflow again manually
- After the job has finished, the report shouldn't appear on the latest run (step 2) but on the previous one from step 1.
Expected behavior
Each generated report should appear on the corresponding run summary page
Additional context
- Using version 1.8.0
- All runs are being triggered manually (workflow_dispatch)
I think this is related to the check run creation function, seems like the checks.create takes as an argument the commit sha, and there is no way to specify the run_id (not sure though)
In an ideal world, there is no need to retest if there was no new changes to the code, but in the real world, I think a lot of us rerun tests esecially when there are fialed tests and a lot of times the tests succeed after that, it would be better if one can check the report on the same run summary
Related to #367 ?
Not sure tbh, on the other issue, the report seems to be generated under the "CodeQL section" as reported by some, and I don't know what's that, in my case the reports are generated under an old run.
I have experienced a similar issue.
When a trigger starts multiple workflows, the report checks appear under the workflow that runs first rather than the workflow to which the checks belong.
I have experienced this issue including when I demoed this tool at NDC Sydney. Looking forward to a fix when you have a moment.
Would be nice to get this fixed, the report ends up in an unrelated workflow run.
https://github.com/EnricoMi/publish-unit-test-result-action/issues/12 has an identical issue and suggests it's caused by https://github.com/orgs/community/discussions/24616.
+1 - Just added the action today and it is exactly what I was looking for, thank you! Same behavior, the job/Test Report is added to workflow based on commit number instead of the workflow run number.
Our team's preference is for the job/Test Report to be added to the current run number, regardless of commit.
I can confirm we see test results appear randomly in other build steps or jobs.
