Improve error formatting for CLI applications

[brendanzab]

Currently I'm not doing much about this. Here are some options

• We could try to follow Rust closely.
• Get inspiration from other languages and go our own way.
• Provide multiple error formatting styles (could help us support other backends, like the Language Server Protocol).

I'd also like to be able to support domain-specific errors, such as those produced by LALRPOP - at least providing an escape hatch to allow them to be easily created!

Inspiration:

Here are some examples of what the output might look like:

Reason

Source code

LALRPOP

Source code

calc.lalrpop:6:5: 6:34: Ambiguous grammar detected

  The following symbols can be reduced in two ways:
    Expr "*" Expr "*" Expr

  They could be reduced like so:
    Expr "*" Expr "*" Expr
    ├─Expr──────┘        │
    └─Expr───────────────┘

  Alternatively, they could be reduced like so:
    Expr "*" Expr "*" Expr
    │        └─Expr──────┤
    └─Expr───────────────┘

  Hint: This looks like a precedence error related to `Expr`. See the LALRPOP
  manual for advice on encoding precedence.

Elm

Source code

Rust

Source code

[brendanzab]

Here's a nice post of Flow's new error formatting: https://medium.com/flow-type/better-flow-error-messages-for-the-javascript-ecosystem-73b6da948ae2

[Marwes]

For LSP there are some things to keep in mind.

• The first line of the message should have enough information that it and the line where the error occurred is enough to fix the problem. This is because editors tend to require additional action to view the rest of the message.

• Colors can't be used to mark important sections of the message. Instead marked locations must be done via ^^^^~~~ etc.

• On the other hand the editor will mark at least the line at which the error occured so drawing out spans in the message might not be as important, unless they involve multiple spans (currently the LSP only supports one span per diagnostic but RLS is experimenting with a way to group multiple diagnostics into one https://github.com/rust-lang-nursery/rls/pull/681).

[brendanzab]

Oh cool. Seems like they are hashing things out with the LSP folks on Microsoft/language-server-protocol#166 as well. 🤔

[brendanzab]

At the moment I'm thinking we'd want to have common Diagnostic struct that tries to stick to the 'semantic model' of a diagnostic, leaving ultimate 'styling' to downstream consumers, like the CLI formatter or LSP. We'd simplify/throw away some info when supporting the LSP for example.

[brendanzab]

Some nice output from Hedgehog https://github.com/hedgehogqa/fsharp-hedgehog/issues/80#issuecomment-312884705