Failing integration test: Socket backpressure

[nothingismagick]

trafficstars

On a fresh clone, I install the dependencies, and find this test failure:

  1) Integration tests
       Socket backpressure
         Should be able to getOutboundBackpressure() on a socket object:

      AssertionError [ERR_ASSERTION]: false == true
      + expected - actual

      -false
      +true
      
      at Context.<anonymous> (test/integration.js:1786:14)

[nothingismagick]

So it seems the backpressure is just flaky. Sometimes when I run the test everything passes, sometimes 2 fail.

L1786:      assert.equal(backpressureHistory[14] > 8, true);
L1834:      assert.equal(backpressureHistory[14] > 6, true);

[jondubois]

I think this maybe be more of a problem with the test case than with the backpressure mechanism. There is a non-deterministic aspect to measuring backpressure which makes it difficult to write these test cases. It may be related to machines having different performance characteristics.

The ranges could be made bigger in the test cases to provide more consistent results on different machines.

It's not possible to make these kinds of integration tests fully deterministic because of the way networks and asynchronous processing works. Also JavaScript's setInterval is not exact.

[nothingismagick]

Yeah - I sort of thought that was the case. I see a couple solutions (if you think its important to have all your tests passing):

• split out this kind of "non-deterministic" tests to their own suite and don't run them on CI
• ask them to run several (5?) times and if in any case all tests pass then consider the suite to have passed

But this all assumes you are interested in having them run in gh-actions (which is free for open source) or drone / jenkins etc. I know I would have more faith in the library moving forward if you were running tests on PRs to Master / Development etc.