errorForStack on external code exceptions not reported and all unit tests still pass.

[chrisribe]

Tell us about your runtime:

QUnit version: 2.15.0 Which environment are you using? (e.g., browser, Node): ChromeHeadless

How are you running QUnit? (e.g., QUnit CLI, Grunt, Karma, manually in browser): Karma

What are you trying to do?

Code that reproduces the problem: Externally loaded code that generates an exception because of an undefined variable case. Does not report the unit test as failed. Even though debug console log shows a js error and prints "FAILED". Total unit tests and final result shows success.

I am using Karma with karma iframes. I tried to create a demo test case but could not reproduce. https://www.npmjs.com/package/karma-iframes

I did however find a place where "detecting and reporting" the issue could be done to flag the issue.

https://github.com/qunitjs/qunit/blob/88e3b74bd4761f94bd40d73c3799ff7e7ffc840a/src/assert.js#L145 Add

        if (!currentTest || (currentTest.errorForStack)) {
          throw new Error("assertion outside test context, in " + sourceFromStacktrace(2));
        }

What did you expect to happen?

UNIT tests should fail if any

What actually happened?

UNIT tests shows success even if js errors are present in console.

[chrisribe]

Any updates on this ?

[smcclure15]

This sounds like a potentially serious issue (false positive), and I'm all for defensive-coding, but IMHO a lack of repro steps gives me pause to action anything and feel confident that we really got it.

Any way to reproduce?

[chrisribe]

I tried to reproduce in a demo project but could not. I had to continue work on another project and stopped looking. I will go over our current setup again and try to get the issue reproduced.

[Krinkle]

Declining for now, but don't hesitate to let us know if you can reproduce it again (either here, happy to re-open! Or as new issue).