multi-reporter example fails

[Hipska]

Hi,

I'm trying to have spec output and JUnit report while running Cypress from command line. But having some issues.

First, it seems that Cypress cannot load mocha-multi-reporters if it is installed globally on the system:

Could not load reporter by name: mocha-multi-reporters

We searched for the reporter in these paths:

• /path/to/project/mocha-multi-reporters
• /path/to/project/node_modules/mocha-multi-reporters

Learn more at https://on.cypress.io/reporters

Second, when running these exact commands:

npm install --save-dev mocha mocha-multi-reporters mocha-junit-reporter
cypress run --reporter mocha-multi-reporters --reporter-options configFile=config.json

Cypress fails just after running the tests with the following stack trace (twice):

TypeError: test.titlePath is not a function
    at /path/to/project/node_modules/mocha/lib/reporters/base.js:227:10
    at Array.forEach (native)
    at Function.exports.list (/path/to/project/node_modules/mocha/lib/reporters/base.js:167:12)
    at Spec.Base.epilogue (/path/to/project/node_modules/mocha/lib/reporters/base.js:344:10)
    at emitOne (events.js:101:20)
    at Runner.emit (events.js:188:7)
    at Reporter.emit (/usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/server/lib/reporter.js:168:55)
    at Object.server.startWebsockets.onMocha (/usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/server/lib/project.js:224:22)
    at Socket.<anonymous> (/usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/server/lib/socket.js:237:36)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at /usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/socket/node_modules/socket.io/lib/socket.js:503:12
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

I can't really tell if this is an Cypress or Mocha issue, so I made the issue on this repository because here are the steps documented and they seem not to work..

[bahmutov]

Hmm, the current repo shows local install of multi-reporters with Cypress v1.0.2 and it is running fine. Can you try same case that fails for you with Cypress v1.0.2 following this repo example? If the difference is in globally installed multiple reporters module, can you make a working example with failure, even as a fork of this project, please?

[Hipska]

I could pinpoint it to a single test file that caused cypress to crash with the given stack trace, but it does not happen when that test file is added to this repo. Still figuring out what could be different..

[Hipska]

Unzip this project and install multi reporters as explained in the docs: webtest.zip Then run cypress and see the crash.

[AyWa]

Can you try with "mocha": "^3.0.0", instead of "mocha": "^4.0.0", ? it seems to work for me (cypress 2.X)

[valscion]

I have also trouble with multi reporters, but cypress doesn't crash for me. Instead, with [email protected] AND with [email protected] I got only one report recorded:

Contents of the junit report with mocha 5.2.0

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="0.861" tests="3" failures="0">
  <testsuite name="Root Suite" timestamp="2018-07-10T12:20:54" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="Window" timestamp="2018-07-10T12:20:54" tests="3" failures="0" time="0.861">
    <testcase name="Window cy.window() - get the global window object" time="0.401" classname="cy.window() - get the global window object">
    </testcase>
    <testcase name="Window cy.document() - get the document object" time="0.241" classname="cy.document() - get the document object">
    </testcase>
    <testcase name="Window cy.title() - get the title" time="0.219" classname="cy.title() - get the title">
    </testcase>
  </testsuite>
</testsuites>

yarn cypress run --reporter mocha-multi-reporters --reporter-options configFile=./cypress-reporter-config.json

The contents of cypress-reporter-config.json were:

{
  "reporterEnabled": "spec, mocha-junit-reporter",
  "mochaJunitReporterReporterOptions": {
    "mochaFile": "multiple-results/results.xml"
  }
}

The versions I had were:

[email protected]
[email protected]
[email protected]
[email protected]

My issue seems to be same as https://github.com/stanleyhlng/mocha-multi-reporters/issues/52 — maybe this is a bug in mocha-multi-reporters?

The spec reporter does show that all tests were ran, though:

[valscion]

Could be that the comment by @kapalkat in https://github.com/stanleyhlng/mocha-multi-reporters/issues/52#issuecomment-405712818 might contain a fix:

This is working as expected. The problem was with configuration. Test-report was overwritten by next test. The solution is to set the > following: cypress/results/test-report-junit.[hash].xml Thanks to that each test has its own unique file with the result.

{
  "reporterEnabled": "spec, mocha-junit-reporter",
  "mochaJunitReporterReporterOptions": {
    "mochaFile": "cypress/results/test-report-junit.[hash].xml",
    "toConsole": true
  }
}

I'm on vacation right now and am able to test this configuration out only after two weeks.

[valscion]

Heh, well better late than never, right? I can verify that my previous comment above this one works, as long as we're using mocha v4.x. It crashed with mocha v6.x