Reaching mocha timeout leads to incorrect coverage (vscode extension testing)

[gnikit]

• Version: v18.7.0
• Platform: Linux x86_64 (should affect all)

This is more of a note rather than an bug report and would probably be more appropriate as a GitHub Discussion if they were enabled.

We just starting using c8 for coverage in our VS Code extension https://github.com/fortran-lang/vscode-fortran-support and what I noticed was that if we hit our timeout in our mocha unittests c8 carries on with the coverage report as if the unittests have run succesully. Maybe it would be beneficial if c8 threw some kind of warning that the coverage might be inaccurate because you reached timeout.

In our case, we had some relatively expensive suiteSetup and suiteTeardown, so even though our tests were passing c8 would produce very incorrect reports, e.g. 40% as opposed to 70%. It took me a while to figure out what was going on given the nightmare that vscode unittesting tends to be.

To reproduce you can checkout our repo on commit https://github.com/fortran-lang/vscode-fortran-support/pull/629/commits/801c40ce4a80af7452931efda1b3f15e2f271ade and

npm ci
npm run coverage

An example where coverage is very incorrect is the file linter-provider.ts