mochawesome-report-generator icon indicating copy to clipboard operation
mochawesome-report-generator copied to clipboard

Published 20 hours ago • verified publisher icon adamgruber

Bug: report considers skipped tests as failed

Open dialex opened this issue 6 years ago • 2 comments
trafficstars

I'm using your report with Cypress. I have one skipped Cypress test:

Screenshot 2019-07-25 at 14 39 35

Expected: either 100% tests passed (green); or 93% passed and 7% tests skipped (grey) Actual: bar shows red and 7% of tests failed, which is not true

It looks to me your report considers skipped tests as failed

dialex avatar Aug 01 '19 07:08 dialex

A skipped test is usually the result of a failed before hook, which the report does consider.

adamgruber avatar Aug 01 '19 16:08 adamgruber

There's a related ongoing discussion on Cypress https://github.com/cypress-io/cypress/issues/3092#issuecomment-468138496

This was not the case, I skipped the test deliberately, not because a hook failed. I would prefer the reporter wouldn't assume things. If the test was skipped, show it as skipped. Simple and transparent.

Because right now you have incoherent data. Your report says at the same time 0% and 7% tests failed. The testing world is all about information and confidence, and incoherent data destroys any confidence in it.

dialex avatar Aug 02 '19 13:08 dialex