haskell
Custom tasty `consoleReporter`
There are two issues with the current consoleReporter (which are both mostly nitpicks and my personal preference).
- It can be hard to find the test case that was failing during a test run (especially on CI)
- The alignment of the test output can be sometimes hard to read (see https://github.com/haskell/haskell-language-server/pull/4591 where this caused an issue for me)
I am proposing that we introduce a custom consoleReporter which fixes both of these "shortcomings".
I have a prototype that makes the output look like this:
This makes the output more terse, but not necessarily easier to read. It is only easier to read, if the test names are too long, like in #4591, which can be generally worked around (by using shorter test names).
Additionally, I would propose a succinct summary at the end of the test execution if the test failed, which should fix my first issue with the consoleReporter.
If we had a custom consoleReporter, this would be relatively easy to add.
- It can be hard to find the test case that was failing during a test run (especially on CI)
Could you try --hide-successes?
- The alignment of the test output can be sometimes hard to read (see https://github.com/haskell/haskell-language-server/pull/4591 where this caused an issue for me)
This is tracked at https://github.com/UnkindPartition/tasty/issues/342, a PR would be welcome.
Could you try --hide-successes?
Personally, I want to see the successes, as it is visual confirmation that the tests are run and that a particular test is run. If I hide successes, I don't see the progress, which is important to me when locally running tests. Even in CI, I like that I can see how long the test execution took, which test is skiped, etc... so I prefer to not --hide-succeses.
Even worse, due to unfortunate stderr output done by HLS and GHC occasionally, the output looks like this:
Note, this output is likely a regression caused by https://github.com/haskell/haskell-language-server/pull/4480