feat(cli): add option or command to output JSON

[dosolkowski-work]

Description

We run this tool as part of our automated build process and report findings in GitLab code quality reports. In order to do this, we need to parse and reformat the output into GitLab's code quality file format. This is difficult when the output format keeps changing.

Suggested Solution

Add a command-line flag to allow us to specify the output format, and support JSON or at least some sort of stable plain-text format. It does not need to specifically be GitLab's code quality format; anything stable we can write a script to translate it.

Help Needed

I'm not sure if I can legally contribute to outside projects with my employment contract, but tools like ESLint and StyleLint support JSON output or customizable output formats.

[tylerd-canva]

@JamieMason are you open to this being contributed as part of the new Rust implementation? Do you have a preference on output format? For example, SARIF would be very convenient.

[JamieMason]

Hey @tylerd-canva, thanks for asking.

I had a rough idea of having a syncpack json command which would output every instance of every dependency, with additional properties to state which package it is in, which version group it is in, its validity and what error code that is etc.

The rough idea was to add a new subcommand branch here

https://github.com/JamieMason/syncpack/blob/2374a4607c067df65e8ba7a64a6078a475452bf8/src/main.rs#L73-L76

a new "effects" module here (a module to take the generated state object of the monorepo and do whatever with it)

https://github.com/JamieMason/syncpack/blob/2374a4607c067df65e8ba7a64a6078a475452bf8/src/effects.rs#L3-L8

and then create that effects module based on eg the lint one

https://github.com/JamieMason/syncpack/blob/2374a4607c067df65e8ba7a64a6078a475452bf8/src/effects/lint.rs#L1-L4

The module would loop over the data and print out lines of JSON.

I'm not familiar with SARIF but if it is a popular standard that makes total sense.

I've taken a break from working on this project but I check in on the issues and should be returning to working on it again in a few weeks.

[tylerd-canva]

I'm not an expert on SARIF but it is a standard for linters to output errors that point to lines of code. We're using it internally for some other linters, it makes it easier to do things like put inline annotations from many different linter sources on PRs on GitHub. Your JSON command sounds a lot more generic than what I was imagining -- my naive approach was going to be to just add something like --format sarif to the syncpack lint command and print out any issues in the SARIF format instead of the default UI.

Here's an example of what you can get on GitHub once you wire it up:

Of course your approach and my approach are not mutually exclusive. Certainly using your proposal I could build some sort of outside wrapper that converts the JSON to SARIF, but I figured I'd offer to contribute the built-in approach since you already had this issue open. Plus since you're working on a Rust rewrite and there's a Serde crate for SARIF, it seemed almost serendipitous.

[JamieMason]

I'm open to any direction, people have mentioned JSON output before but the use cases were never clear to me so I'd been guessing about what they'd be used for. My hunch was that outputting some flat structure would support the use cases as that output could be piped into jq, grep etc to do whatever they needed.

~I've only very quickly skim read it but it looks like serde SARIF could replace serde JSON as what syncpack uses internally(?) If it works mostly the same but offers support for tracing back the line numbers that's a huge upgrade.~

One thing I'm missing in the Rust implementation is #241 is regressed and I can't find anything in Rust that maintains the original formatting of the parsed in JSON file, ~do you know if SARIF does that?~

[tylerd-canva]

I'm not yet familiar with Syncpack internals, but I'll have a look. I've seen a similar problem before.

I wasn't necessarily thinking about using SARIF as an internal intermediate format -- I would think that Syncpack would have it's own types so it wouldn't need JSON or anything else except for output (or input like for the config file).

[JamieMason]

Thanks, yeah I hadn't seen an example of a SARIF file but have now and it's not the same thing – ignore what I said there. I had this impression it was some kind of JSON AST which it isn't.

[tylerd-canva]

I'm still curious about the format-preservation regression -- to avoid mixing two discussions in this thread, do you want to make a separate issue to track that one? Or maybe there already is one?

[kimroen]

Just a little note that I came here after trying syncpack lint --json and syncpack lint --format=json and not having any success 😄

My use case was that I wanted to store the output in a file on CI so that I could parse it in a later job and report back to the user if something was wrong (alongside other checks, like linting and test results). I can do that with the text output too, but having it available in a structured way allows me to adjust how it looks and surface the most relevant bits for us.

[JamieMason]

Work has started on this. Currently it will be a new command syncpack json which outputs the status of every instance as a line of JSON, to be likely piped into jq or similar to be filtered etc.

The usual CLI filters will be supported:

# only AWS SDK
syncpack json --dependencies '@aws-sdk/**'
# only missing .version properties, or invalid specifiers
syncpack json --specifier-types missing,unsupported

This is the shape of the data at the moment

https://github.com/JamieMason/syncpack/blob/483804ac4ce139ae75b621e9f1f2445aaa9304ed/npm/syncpack.ts#L171-L189

EDIT: It will exit with 0 or 1 in the same way as syncpack list and syncpack lint do.

[JamieMason]

Released in 14.0.0-alpha.18.

SARIF I can't currently support, but hopefully in the future some Rust crate I can work with can help form a foundation for that.