Errors reported in different test cases

[countergram]

I don't have a minimal example of this factored out yet, but in our moderate sized integration suite (~15 describes, ~300 tests), with mostly async tests, I am noticing errors being reported with the wrong test case name, or multiple times with different test case names (which can be seen because the traceback is the same under different names).

[1999]

Hey @countergram, thank you for posting this out. Unfortunately without any example this is hard to fix although it seems like you're reusing some variables between different test suites with mocha. Can you spend some time to decrease this suite to reproducable example?

[1999]

@countergram ping

[countergram]

Hi. Here is a minimal example with code + output. All test cases are async using "done", and exactly one failing case ends up being reported multiple times.

describe('the error one', () => {
  it('oops', done => {
    setTimeout(() => {
      throw new Error('this is my error');
    }, 500);
  });
});

for(let descNum = 0; descNum < 5; descNum++) {
  describe('describe #' + descNum, () => {
    for(let itNum = 0; itNum < 20; itNum++) {
      it('case #' + itNum, done => {
        setTimeout(done, Math.random() * 1000);
      });
    }
  });
}

Output varies, but here is some sample output:

mocha-parallel-tests --max-parallel 5 test.js



  the error one
    1) oops


  describe #3
    ✓ case #0 (112ms)
    2) case #1
    ✓ case #2 (612ms)
    ✓ case #3 (423ms)
    ✓ case #4 (311ms)
    ✓ case #5 (576ms)
    ✓ case #6 (494ms)
    ✓ case #7 (797ms)
    ✓ case #8 (136ms)
    ✓ case #9 (198ms)
    ✓ case #10 (786ms)
    ✓ case #11 (41ms)
    ✓ case #12 (534ms)
    ✓ case #13 (472ms)
    ✓ case #14 (467ms)
    ✓ case #15 (484ms)
    ✓ case #16 (289ms)
    ✓ case #17 (293ms)
    ✓ case #18 (376ms)
    ✓ case #19


  describe #2
    3) case #0
    ✓ case #1 (370ms)
    ✓ case #2 (118ms)
    ✓ case #3 (710ms)
    ✓ case #4
    ✓ case #5 (345ms)
    ✓ case #6 (604ms)
    ✓ case #7 (603ms)
    ✓ case #8 (75ms)
    ✓ case #9 (448ms)
    ✓ case #10 (146ms)
    ✓ case #11 (118ms)
    ✓ case #11 (118ms)
    ✓ case #13 (785ms)
    ✓ case #14 (485ms)
    ✓ case #15 (704ms)
    ✓ case #16 (352ms)
    ✓ case #17 (601ms)
    ✓ case #18 (267ms)
    ✓ case #19


  describe #4
    4) case #0
    ✓ case #1 (185ms)
    ✓ case #2 (559ms)
    ✓ case #3 (111ms)
    ✓ case #4 (395ms)
    ✓ case #5 (738ms)
    ✓ case #6 (300ms)
    ✓ case #7 (194ms)
    ✓ case #8 (973ms)
    ✓ case #9 (948ms)
    ✓ case #10 (200ms)
    ✓ case #11
    ✓ case #12 (434ms)
    ✓ case #13 (622ms)
    ✓ case #14 (809ms)
    ✓ case #15 (247ms)
    ✓ case #16 (131ms)
    ✓ case #17 (649ms)
    ✓ case #18 (293ms)
    ✓ case #19


  describe #0
    5) case #0
    ✓ case #1 (359ms)
    ✓ case #2 (127ms)
    ✓ case #3 (214ms)
    ✓ case #4 (597ms)
    ✓ case #5 (991ms)
    ✓ case #6 (525ms)
    ✓ case #7 (643ms)
    ✓ case #8 (203ms)
    ✓ case #9 (383ms)
    ✓ case #10 (702ms)
    ✓ case #11 (361ms)
    ✓ case #12 (362ms)
    ✓ case #13
    ✓ case #14 (812ms)
    ✓ case #15 (688ms)
    ✓ case #16 (504ms)
    ✓ case #17 (316ms)
    ✓ case #18
    6) case #19


  describe #1
    ✓ case #0 (440ms)
    7) case #1
    ✓ case #2 (246ms)
    ✓ case #3 (784ms)
    ✓ case #4 (626ms)
    ✓ case #5 (85ms)
    ✓ case #6 (972ms)
    ✓ case #7 (513ms)
    ✓ case #8 (877ms)
    ✓ case #9 (255ms)
    ✓ case #10 (663ms)
    ✓ case #11 (668ms)
    ✓ case #12 (194ms)
    ✓ case #13 (370ms)
    ✓ case #14 (415ms)
    ✓ case #15 (717ms)
    8) case #16
    ✓ case #17
    ✓ case #18
    9) case #19


  92 passing (5s)
  9 failing

  1) the error one oops:
     Uncaught Error: this is my error
      at Timeout.setTimeout (test.js:4:13)

  2) describe #3 case #1:
     Uncaught Error: this is my error
      at Timeout.setTimeout (test.js:4:13)

  3) describe #2 case #0:
     Uncaught Error: this is my error
      at Timeout.setTimeout (test.js:4:13)

  4) describe #4 case #0:
     Uncaught Error: this is my error
      at Timeout.setTimeout (test.js:4:13)

  5) describe #0 case #0:
     Uncaught Error: this is my error
      at Timeout.setTimeout (test.js:4:13)

  6) describe #0 case #19:
     Uncaught TypeError: Cannot set property 'state' of undefined
  

  7) describe #1 case #1:
     Uncaught Error: this is my error
      at Timeout.setTimeout (test.js:4:13)

  8) describe #1 case #16:
     Uncaught TypeError: Cannot set property 'state' of undefined
  

  9) describe #1 case #19:
     Uncaught TypeError: Cannot set property 'state' of undefined