Socket timeouts POSTing to /write-coverage when running tests in CI mode

[cah-jordan-smith03]

Ember: 3.28.6 ember-cli-code-coverage: 2.0.0 ember-cli-mirage: 2.3.2 ember-qunit: 6.2.0 qunit: 2.19.4 Testem: 3.10.1 Project type: classic project using ember-cli-build.js Node version: 16.13.0 OS: Mac OSX Ventura 13.2.1

Good afternoon,

I have configured the project that I am working on to leverage ember-ci-code-coverage. If I run in test server mode using COVERAGE=true ember t -s, test coverage is working as expected. However, if I run in CI mode using COVERAGE=true ember t, the sendCoverage() method fails with the following error:

BadRequestError: request aborted
    at IncomingMessage.onAborted (project-folder/node_modules/raw-body/index.js:245:10)
    at IncomingMessage.emit (node:events:390:28)
    at IncomingMessage._destroy (node:_http_incoming:179:10)
    at _destroy (node:internal/streams/destroy:102:25)
    at IncomingMessage.destroy (node:internal/streams/destroy:64:5)
    at abortIncoming (node:_http_server:596:9)
    at socketOnClose (node:_http_server:590:3)
    at Socket.emit (node:events:402:35)
    at TCP.<anonymous> (node:net:687:12)

1..791
# tests 791
# pass  766
# skip  25
# todo  0
# fail  0

# ok

test/test-helper.js

QUnit.on('runEnd', async function() {
  forceModulesToBeLoaded();
  // exception is thrown here. If I invoke from Qunit.on('suiteEnd') instead, only the final callback fails to POST
  await sendCoverage();
});

If I create a new project with few/no tests or if I run only a small subset of the tests (acceptance), I never get this error. This seems to be a conflict where Testem has already exited the ExpressJS app from the testemMiddleware() function.

Is running test coverage in CI mode currently supported by this project and if so, is there any recommendation outside of the existing documentation to get it working? I found some examples of using Testem and the legacy istanbul directly but would prefer to avoid that if possible.

EDIT: The more that I looked into this, I am convinced that the root cause is an issue with Testem. https://github.com/testem/testem/issues/1577

[tniezurawski]

@cah-jordan-smith03 Do I understand correctly that QUnit.done:

QUnit.done(async function() {
  forceModulesToBeLoaded();
  await sendCoverage();
});

doesn't work for you either?

[cah-jordan-smith03]

@tniezurawski I should have specified in the original post that I ran into the same issues with Qunit.done before trying Qunit.on('runEnd').

[kategengler]

Before #332 the addon called sendCoverage itself, like so:

if (typeof Testem !== 'undefined' && Testem.afterTests) {
  //eslint-disable-next-line no-undef
  Testem.afterTests(function (config, data, callback) {
    sendCoverage(callback);
  });
} else if (typeof QUnit !== 'undefined') {
  //eslint-disable-next-line no-undef
  QUnit.done(function () {
    sendCoverage();
  });
} else if (typeof Mocha !== 'undefined' && typeof after !== 'undefined') {
  //eslint-disable-next-line no-undef
  after(function (done) {
    sendCoverage(done);
  });
} else {
  console.warn(
    'No testing framework found for ember-cli-code-coverage to integrate with.'
  );
}

I suspect that when running in CI, Testem.afterTests is the correct hook to use, but QUnit.done is the correct hook for running in-browser.

[cah-john-ryan]

For an interim solution, please see: https://github.com/ember-cli-code-coverage/ember-cli-code-coverage/issues/420#issuecomment-2332270042