[RFR] Print test failures in GH Actions

[jtotoole]

[pypt]

Is this PR (or the other one with a similar name) something to look at already, or is it still work in progress?

[jtotoole]

Not ready for prime time yet; I'm facing a stumbling block that I figured we could discuss in more detail tomorrow. Specifically, the awk command I added as a new build step to print just the test failures seems unable to interact properly with joblog.txt.:

The GH Actions logs have nothing to say on the subject, so I'm at a loss.

Closed the other PR related to this as I think the approach here is more likely to be useful.

[pypt]

I think GHA doesn't run step n+1 if step n has failed, so you might want to either make the "Run tests" step exit with a zero error code, merge the "Run tests" and "Print test failures" into a single step, or (perhaps better approach) add some conditionals.

Also, running the full test suite takes a whole lot of time so you might consider temporarily removing a bunch of tests from the codebase just to make the GHA run finish faster. Later you can just git revert the commit.

[jtotoole]

Finally got this working as intended—see the "List errors and test failures" step in this build

[jtotoole]

ready for review!

[pypt]

Merged in master to trigger a rebuild (the old one has expired).

[pypt]

So printing failures as GitHub Actions annotations didn't work?

[jtotoole]

I initially abandoned that approach because the GitHub Checks API will only publish annotations on test files that have been changed in the PR. There's a marketplace action that will annotate everywhere if we first create a test summary file in JUnit XML format, but that seemed like rather high-hanging fruit compared to this approach. That said, I'm happy to give the JUnit thing a shot if you think it'd be better and not wildly complicated.