Varying results for same test

[byumark]

I am running the following test multiple times in a row:

describe('leaks', function() {
  it('leak test', function() {
    this.timeout(100000)

    return Leakage.iterate.async(async () => {
      return myFunction()
    })
  })
})

Sometimes it passes and other times I get something like this:

MemoryLeakError: Heap grew on 6 subsequent garbage collections (180 of 180 iterations) by 175 kB.

  Iterations between GCs: 30

  Final GC details:
  [   19.4 kB] [+ 123x] [-  84x] Array
  [     14 kB] [+   5x] [-   1x] Code
  [       0 B] [+   1x] [-   1x] Immediate
  [       0 B] [+   2x] [-   2x] Native
  ... (15 more)

Should it consistently pass or fail rather than be intermittent? Thanks.

[andywer]

Hey @byumark. That's quite tricky and depends on a bunch of factors: Primarily your tested code and the node version you use.

In general changes to garbage collection in recent node.js versions seem to have made it harder to obtain consistent behavior. Not sure how much that effectively impacts the tests…