metatests icon indicating copy to clipboard operation
metatests copied to clipboard

Report on the test.rejects failure not working

Open KLarpen opened this issue 1 year ago • 3 comments

Is your feature request related to a problem? Please describe.

I had encounter an issue when trying to look intended test fail results of Impress. Way to reproduce:

  1. Open impress folder in terminal.
  2. Open editor with file test/procedure.js
  3. In the assertion at lines 137-140 of the test metatests.testAsync('lib/procedure timeout'
  await test.rejects(
    async () => procedure.invoke({}, { waitTime: 150 }),
    new Error('Timeout of 100ms reached'),
  );

change the value of waitTime to 50. So the assertion must failed due to test configuration. 4. Run npm t 5. Instead of failed test report there is an error from reporting logic of metatest package itself (more specifically from one of its dependency tap-mocha-reporter)

Node v20.9.0 (v8 11.3.244.8-node.16):
Certificate request self-signature ok
subject=CN = localhost
lib/application ..................................... 21/21
lib/deps .............................................. 8/8
lib/procedure ....................................... 24/24
lib/procedure validate ................................ 4/4
schemas/contracts ..................................... 1/1
lib/api load ........................................ 22/22
lib/place ........................................... 18/18
schemas/config ........................................ 4/4
lib/bus ............................................. 11/11
lib/cert .............................................. 4/4
lib/code ............................................ 11/11
lib/procedure timeout ../node_modules/tap-mocha-reporter/lib/runner.js:293
    return stack.trim().split('\n').map(function (line) {
                 ^

TypeError: stack.trim is not a function
    at reviveStack (/node_modules/tap-mocha-reporter/lib/runner.js:293:18)
    at getError (/node_modules/tap-mocha-reporter/lib/runner.js:332:87)
    at emitTest (/node_modules/tap-mocha-reporter/lib/runner.js:280:17)
    at Parser.<anonymous> (/node_modules/tap-mocha-reporter/lib/runner.js:219:5)
    at Parser.emit (node:events:514:28)
    at Parser.emit (node:domain:488:12)
    at Parser.emit (/node_modules/minipass/index.js:483:23)
    at Parser.emitAssert (/node_modules/tap-parser/index.js:864:10)
    at Parser.emitResult (/node_modules/tap-parser/index.js:733:10)
    at Parser.plan (/node_modules/tap-parser/index.js:417:10)
  1. The error not only failing the report but breaking execution of subsequent test cases, e.g. 'lib/procedure validate async'.

Describe the solution you'd like

Introduce better reporters instead of tap-mocha-reporter as it was described in #265

Describe alternatives you've considered

Please propose temporary workaround. I had tried to investigate function reviveStack in /node_modules/tap-mocha-reporter/lib/runner.js:293 by adding console.debug({ stack, stackFile: stack.file }). It seems that contract of stack that the function receives has been changed and even not stable to rely on: two internal subsequent calls resulting in string and object

{
  stack: 'async ImperativeTest.func (/test/procedure.js:137:3)\n',
  stackFile: undefined
}
{
  stack: { file: '/test/procedure.js' },
  stackFile: '/test/procedure.js'
}

Unfortunately still had not found the temporary solution.

Additional context

OS: macOS 14.1.2 Node: 20.9.0, 18.18.2 Impress: 3.0.13 Metatests: 0.8.2

KLarpen avatar Dec 17 '23 12:12 KLarpen