utap icon indicating copy to clipboard operation
utap copied to clipboard

More useful output formatting for errors?

Open fenomas opened this issue 1 year ago • 0 comments

Hey, this is really useful and I'm enjoying using it. Would it be possible, however, to do a little formatting of the output when a test fails? The default TAP output is mostly noise, and tends not to include what one wants.

As a baseline, one would hope to see just the basics - the message and line number of the test that failed, and maybe the console output from the test script. In contrast, typical utap output looks like:

andy:someProject > node --test | npx utap
  ✔  /Users/andy/dev/someProject/test/conformers.js
  ✔  /Users/andy/dev/someProject/test/core.js
  ✔  /Users/andy/dev/someProject/test/internals.js
  ❌ /Users/andy/dev/someProject/test/parsers.js
      duration_ms: 114.011042
      failureType: 'subtestsFailed'
      exitCode: 1
      stdout: |-
        TAP version 13
        # Subtest: parsing raw source
        ok 1 - parsing raw source
          ---
          duration_ms: 2.119305
          ...
        # Subtest: basic element parsing
        ok 2 - basic element parsing
          ---
          duration_ms: 1.253776
          ...
        # Subtest: element parsing of metadata
        not ok 3 - element parsing of metadata
          ---
          duration_ms: 1.160601
          failureType: 'testCodeFailure'
          error: 'Parsing element reqs'
          code: 'ERR_ASSERTION'
          stack: |-
            TestContext.<anonymous> (file:///Users/andy/dev/someProject/test/parsers.js:167:5)
            Test.runInAsyncScope (node:async_hooks:203:9)
            Test.run (node:internal/test_runner/test:511:25)
            Test.processPendingSubtests (node:internal/test_runner/test:277:27)
            Test.postRun (node:internal/test_runner/test:595:19)
            Test.run (node:internal/test_runner/test:539:10)
            async Test.processPendingSubtests (node:internal/test_runner/test:277:7)
          ...
        1..3
        # tests 3
        # pass 2
        # fail 1
        # cancelled 0
        # skipped 0
        # todo 0
        # duration_ms 9.736368

      stderr: ''
      error: 'test failed'
      code: 'ERR_TEST_FAILURE'
  ✔  /Users/andy/dev/someProject/test/util.js

4 passed
1 failed

I guess this is basically the stdout of node --test? But as you can see it's virtually all noise - info about tests that didn't fail, durations, line numbers for node internals, etc. The output of utap would be a lot more useful if most or all of the extraneous stuff could be hidden.

Additionally, the arguments passed to the failed assertion aren't shown. E.g. if the test failure happened in an assertion like strictEqual(foo, bar) then ideally one would hope to see what values foo and bar had. (Though I guess TAP may not expose such information...)

Would improvements like the above be easy/feasible?

Thanks!

fenomas avatar Mar 10 '23 06:03 fenomas