Output format selection and improved automation formats

[lloeki]

Steep outputs pretty errors for humans to consume:

Steepfile:4:2: [error] Type `::Steep::Project::DSL` does not have method `check`
│ Diagnostic ID: Ruby::NoMethod
│
└   check '.'
    ~~~~~

Steepfile:5:2: [error] Type `::Steep::Project::DSL` does not have method `check`
│ Diagnostic ID: Ruby::NoMethod
│
└   check 'Steepfile'
    ~~~~~

But when plugging Steep into tools it's possible yet unwieldy to parse... I did that for ALE and was able to get a surprising amount of information: start/end of multiline message, file, row, column start, severity, message, code, column end via length from the squiggly underlining. Although it works (and was fun) it's also brittle in the sense that if the pretty output changes it's going to break.

Sure there's langserver but in this specific case LSP doesn't quite cut it, and in some automation scenarios langserver just doesn't make sense.

It would be quite nice if steep check would accept a --format parameter, defaulting to pretty but also having stable programmatic formats such as:

• json (like e.g. Rubocop has):

  {
    "metadata": {
      "steep_version": "1.2.3",
      "ruby_engine": "ruby",
      "ruby_version": "3.2.2",
      "ruby_patchlevel": "53",
      "ruby_platform": "arm64-darwin22"
    },
    "files": [
      {
        "path": "lib/foo.rb",
        "diagnostics": []
      },
      {
        "path": "Steepfile",
        "diagnostics": [
          {
            "severity": "error",
            "message": "Type `::Steep::Project::DSL` does not have method `check`",
            "diagnostic_id": "Ruby::NoMethod",
            "location": {
              "start_line": 5,
              "start_column": 4,
              "last_line": 5,
              "last_column": 8,
              "length": 4,
              "line": 5,
              "column": 4
            }
          },
          .......... and so on
        ]
      }
      .......... and so on
    ],
    "sigs": [
      {
        "path": "sig/foo.rbs",
  
      }
    ],
    "summary": {
      "diagnostic_count": 3,
      "sig_file_count": 5,
      "target_file_count": 10,
      "inspected_file_count": 10
    }
  }
  

  I made up the sigs vs files thing, dunno if it is relevant or not, maybe it should just all go into files.

• single line, grep/rg/rspec-like:

  <path>:<line>:<start column>[-< last column >]: <diagnostic id>: "<message>": <code excerpt>
  

  (I think the above should be unambiguously parseable fairly easily, is still somewhat human readable, and is trivially grep-able)

  e.g:

  # line + column
  Steepfile:5:4: Ruby::NoMethod: "Type `::Steep::Project::DSL` does not have method `check`": check 'Steepfile'
  
  # line + start-end column
  Steepfile:5:4-8: Ruby::NoMethod: "Type `::Steep::Project::DSL` does not have method `check`": check 'Steepfile'
  
  #  (dunno if it makes sense for steep to have multiline errors, but let's pretend. note that in this case end col can be < start col)
  # start+end line + start+end column
  Steepfile:5-7:4-2: Ruby::NoMethod: "Type `::Steep::Project::DSL` does not have method `check`": check 'Steepfile'
  

[soutaro]

Hi!

Currently, you can use --save-expectations to generate the reports in YAML format.

I'm also open to add another output format support. YAML can be relatively easily implemented on the top of the expectations output feature.

I think the location can be represented with four-attributes in structured formats -- JSON or YAML.

[lloeki]

I think the location can be represented with four-attributes in structured formats

Oh yeah, I just took Rubocop's outpout directly and tweaked it a bit to sort of make sense in a Steep+RBS context. I was a bit surprised by all these apparently redundant values! I don't know if they're legacy carried over for backwards compatibility or if they can differ in certain corner cases.