bevy icon indicating copy to clipboard operation
bevy copied to clipboard

Add json output flags to our CI tool

Open Brezak opened this issue 1 year ago • 3 comments

This is part 1 of what I'm planing to be a 3 part project to improve how our CI reports errors.

  • Part 1 (this) will introduce a way to output structured error messages which other tools may ingest.
  • Part 2 will be a refactor of how we handle commands so they can do more than invoke the shell.
  • Part 3 will be a command that ingests the structured output from other commands and spits out GitHub workflow commands for our CI.

Objective

Add a way for our CI tool to output messages in a structured form usable by other tools.

Solution

Most everything we currently use in our tools can be configured to output [json](https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages). So we just need to process it a bit and output effectively that.

The only thing that can't do json are our compile fail tests and formatting. Formatting can't do it because the check flag for rustfmt doesn't support json output. The compile fail tests can't do it because they use a custom test harness. We'd maybe be able to switch them to the standard cargo harness. We would likely loose the ability to show stderr diffs which is less than ideal but they don't work right now so it isn't much of a blocker.

The result is that everything that supports json output gets a emit_json flag and I've refactored how we invoke our commands so we can parse the json output they now output.

Testing

The code was linted through our CI tool using the clippy subcommand using it with and without the emit_json flag,

cc @BD103

Brezak avatar May 02 '24 20:05 Brezak

~I'd hold of on merging this until part 2 is ready as this currently removes the colored messages from our CI logs. From personal experience, this makes skimming them for error messages about 3 times as annoying. Part 2 should fix this.~

Brezak avatar May 02 '24 20:05 Brezak

Update. We don't loose color. We use a env variable to force cargo to always output color.

Brezak avatar May 02 '24 23:05 Brezak

Oh and also make sure to mark the PR as ready for review, since you cannot merge a draft. :)

BD103 avatar May 05 '24 13:05 BD103