Optionally support emitting errors and test failures using Github Actions reporter format (like SwiftLint)

[orj]

We run our Xcode builds on GitHub actions and use xcbeautify to make the logs better. However if a test fails or there is a compile error we have to dig through the logs to find the issue. It is possible for tools to emit log messages in a format that Github Actions can interpret.

See https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#logging-commands

SwiftLint has a --reporter option that causes it to emit issues in this format (https://github.com/realm/SwiftLint/blob/master/Source/SwiftLintFramework/Reporters/GitHubActionsLoggingReporter.swift) this makes running SwiftLint in Github Actions work really well.

It would are great it xcbeautify also had the option of reporting errors like this.

[thii]

Thanks for reporting. Just a note that I’ve seen this. I’ll take a look when I have time to work on this again.

[cpisciotta]

Hi @thii! Are you still looking for help with this? Are there any plans (how) to implement this?

I'd like to help where I can to get this merged, so I'd appreciate some context on the state of this request.

[cpisciotta]

I've created this draft PR: #107.

I'd appreciate some initial reviews and guidance on how to iterate the proposed implementation.

I've tested it on my project; right now, I'm noticing some errors around duplicate compiler warning/error output and test formatting. I'm also seeing annotations related to 3P dependencies- not sure if that's helpful and/or desired.

[finestructure]

This looks really interesting but I'm not quite sure if I'm using this new renderer correctly. Is there an example somewhere what this would look like in a GH action with a test failure?

I've triggered an action with a failing test and the result looks like this:

This is what this same error looked like before this change:

Sure, the error is a bit more obvious but given the length of our logs (thousands of lines) this change isn't what I was hoping for.

I suspect the github-actions format error report should be showing up elsewhere in the GH action UI but I can't see any difference between the two runs. Is perhaps something in our setup preventing this from properly propagating?

[finestructure]

@orj @cpisciotta @thii Apologies for the direct ping but you might not get notified for comments on closed issues. Would one of you have any guidance regarding my comment above?

[cpisciotta]

Hey @finestructure! Sorry for the delayed response! You'll find the annotations in the summary section of a GitHub Actions run.

In your case, it's here. Annotations that tie to a specific file and line number will show as an inline comment; in your case, there's an example here.

I've been meaning to publish better docs on this, and I will very soon! I've been working with Apple to resolve some issues that I found while doing this work, so some of it is still "under development." I'd also like to iterate on the format of the console logs, since it was a first pass at a very different format of the pre-existing renderer.

Let me know how else I can help, and please provide any feedback if you have any!

[cpisciotta]

Let me know if you're not seeing the annotations via those links. They seem to point to different examples than I wanted.

[finestructure]

Hey @finestructure! Sorry for the delayed response! You'll find the annotations in the summary section of a GitHub Actions run.

Oooh... I've been so used to click through this screen to get to the details that I don't even look at what's there anymore 🤦‍♂️😄

This is a great change, it'll make dealing with test features so much easier. Thank you! ❤️

[finestructure]

BTW, inside GH action runs GITHUB_WORKFLOW is set. So this mode could even be the default and use the normal ANSI colour terminal mode if it isn't set and the annotations otherwise. Just a thought!