MagicMirror
MagicMirror copied to clipboard
e2e tests are instable
No new problem, our e2e test are failing unrelated to the tested code.
Was already discussed in some PR's so I decided to make an issue for this.
My observations:
-
not related to native
fetch
, was the same withnode-fetch
before -
most fails are with node
v20
(but not exclusive) -
seems to be a timing problem, very hard to reproduce locally
-
looks like different tests interfere, maybe the next one is started before the predecessor stopped
-
some strange errors are mocked in
tests/e2e/helpers/mock-console.js
:const mockError = (err) => { if ( err.includes("ECONNREFUSED") || err.includes("ECONNRESET") || err.includes("socket hang up") || err.includes("exports is not defined") || err.includes("write EPIPE") || err.includes("AggregateError") || err.includes("ERR_SOCKET_CONNECTION_TIMEOUT") ) { jest.fn(); } else { console.dir(err); } }; global.console = { log: jest.fn(), dir: console.dir, error: mockError, warn: console.warn, info: jest.fn(), debug: console.debug };
So removing mockError here will show these errors and may can lead to the real error