MagicMirror icon indicating copy to clipboard operation
MagicMirror copied to clipboard

e2e tests are instable

Open khassel opened this issue 10 months ago • 4 comments

No new problem, our e2e test are failing unrelated to the tested code.

Was already discussed in some PR's so I decided to make an issue for this.

My observations:

  • not related to native fetch, was the same with node-fetch before

  • most fails are with node v20 (but not exclusive)

  • seems to be a timing problem, very hard to reproduce locally

  • looks like different tests interfere, maybe the next one is started before the predecessor stopped

  • some strange errors are mocked in tests/e2e/helpers/mock-console.js:

    const mockError = (err) => {
      if (
        err.includes("ECONNREFUSED") ||
        err.includes("ECONNRESET") ||
        err.includes("socket hang up") ||
        err.includes("exports is not defined") ||
        err.includes("write EPIPE") ||
        err.includes("AggregateError") ||
        err.includes("ERR_SOCKET_CONNECTION_TIMEOUT")
      ) {
        jest.fn();
      } else {
        console.dir(err);
      }
    };
    
    global.console = {
      log: jest.fn(),
      dir: console.dir,
      error: mockError,
      warn: console.warn,
      info: jest.fn(),
      debug: console.debug
    };
    

    So removing mockError here will show these errors and may can lead to the real error

khassel avatar Sep 18 '23 18:09 khassel