Report shows failure of previous build

[jameskleeh]

https://github.com/micronaut-projects/micronaut-core/runs/1118820767?check_suite_focus=true

This build failed once, then was re-executed and the test report shows the old failure. All builds passed and the test-report task was skipped in all cases.

[jmisur]

I noticed that you are not running the report if there is no failure: https://github.com/micronaut-projects/micronaut-core/blob/6e2f1d1b180ed56b31590e609503468c3e7c2490/.github/workflows/gradle.yml#L43. Maybe that's the case. Otherwise it would overwrite previous report with success.

[famod]

@jmisur Do you generally suggest to use always()? Thing is, I might add your awsome action to https://github.com/quarkusio/quarkus but the CI there is really heavy, with many jobs in one workflow. I'm thinking about adding a test report only if the mvn call failed (it fails on the first test failure).

[jmisur]

What I suggest is to decouple the test reporting from build process. So even if the build fails for any reason, you want to see the test reports (if any) anyway. So we prefer to set --fail-at-end or -Dmaven.test.failure.ignore=true to the build process itself so it doesn't stop at first failed test, but continues till the end and then report with this action. Sometimes projects are also misconfigured and no tests would run, then this action would fail because it would expect at least a single test.

[famod]

@jmisur Thanks for your answer. The key question I'm having whether the action must run always to have a consistent report for each run/build?

[jmisur]

It's preferable. Otherwise you don't know if the report matches the build. But of course rerun of the same commit shouldn't change anything (unless you have flaky tests).

[nineinchnick]

https://github.com/ScaCap/action-surefire-report/pull/81 should also help here