CI Test for Windows failing sometimes on tests with setTimeout

[itsyme]

Please confirm that you have searched existing issues in the repo

Yes, I have searched the existing issues

Any related issues?

No response

Tell us about your environment

MacOS

MarkBind version

4.1.0

Describe the bug and the steps to reproduce it

Sometimes when pushing PRs, the CI / test for windows fails in tests with setTimeout (e.g. tests for Quiz and ScrollTopButton). Rerunning the tests fixes it, however I was wondering if it could be an OS specific issue that we could fix as it only fails for the windows CI / test.

Expected behavior

No response

Anything else?

No response

[EltonGohJH]

It will be good to show pictures of the failing test as the github actions logs will eventually expire. I think for now you can provide the link to the failed github action too (since it takes q a while to expire)

[lhw-1]

Quite timely - I ran across this error in my newest PR #2239 :sweat_smile:

It does seem like it only fails on the Windows CI, as all tests passed for the Ubuntu CI and the Mac CI. Note that the test suites all passed locally (you can pull the branch for PR #2239 and run npm run test to verify). ~~I am quite confident that re-running the tests will make it work (since it works on local, Ubuntu CI, and Mac CI), but I will leave alone it for now to help highlight the issue.~~ I have re-run the tests, and all the tests pass.

Here are screenshots of the relevant errors (warning: quite lengthy), and if you still need more information, you can find the raw logs here.

[tlylt]

@itsyme can you put up a PR to disable those tests for now until the investigation is over?

[itsyme]

Not sure if this is related to this issue but this test on #2311 was only failing on the MacOS tests, was wondering if this could be a issue with our CI rather than the use of setTimeout

[tlylt]

Not sure if this is related to this issue but this test on #2311 was only failing on the MacOS tests, was wondering if this could be a issue with our CI rather than the use of setTimeout

I think this might be a separate issue, possibly due to the differences in the machine setup that the CI process is running on (hence some process takes longer than expected).

The error stated is:

@markbind/core: thrown: "Exceeded timeout of 5000 ms for a test. @markbind/core: Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

We are not currently using setTimeout in this failed test:

test('includeFile detects cyclic references for static cyclic includes', async () => {

I think the fix for this failed test should be to find out how to configure and define a higher overall timeout value for Jest. But we can KIV this as it seems to be quite rare.

[EltonGohJH]

I was looking at the Github actions. And I think one of them failed in Nov 2023 due to this. It seems that this is still prevalent. So, I think anyone can take this up if they want to.

• [ ] Increase timeout for jest tests. Also, investigate and see how much we need to increase. (Note: timeout prevents test suite for running too long and if we increase it by too much => may allow for test that is super long to exist)
• [ ] Monitor and close this issue when we no longer see windows ci test failing due to this.