formaat-design
Tests on PinField sometimes fail (source code)
Describe the bug
Probably about 50% of the times, the tests from yarn test:unit fail for the PinField component. The errors have always been the same and have been occurring for the last several minor updates on the Reshaped source. Basically since the PinField was implemented.
Expected behavior The tests to stop failing.
Screenshots
Consecutive screenshots as I can't fit it all in one screenshot.
Can you provide some additional information about the environment? Like the installed package versions in case they were changed, node version, etc? I'm trying to reproduce this error but it still passed every single time for me
Apart from the package version change from this issue, I've not changed any other package versions.
Node version: v22.5.1 Yarn version: 1.22.22
I've managed to get to the same error in about 5 test runs. So I can't find a pattern yet.
I've switched my node version to v20.16.0 and it failed on the second try, although I don't think it's from the node version.
Let's try debugging this remotely if you're ok with that since I still can't reproduce it locally. Does replacing the test case with the following change anything for you?
test("handles keyboard navigation", async () => {
render(
<Reshaped>
<PinField name={fixtures.name} />
</Reshaped>
);
const elInput = screen.getByRole<HTMLInputElement>("textbox");
await act(() => {
elInput.focus();
});
expect(elInput.selectionStart).toEqual(0);
await userEvent.keyboard("1");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(1);
});
await userEvent.keyboard("234");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(3);
expect(elInput.selectionEnd).toEqual(4);
});
await userEvent.keyboard(
"{ArrowLeft}{/ArrowLeft}{ArrowLeft}{/ArrowLeft}{ArrowLeft}{/ArrowLeft}"
);
await waitFor(() => {
expect(elInput.selectionStart).toEqual(0);
expect(elInput.selectionEnd).toEqual(1);
});
await userEvent.keyboard("{ArrowRight}{/ArrowRight}{ArrowRight}{/ArrowRight}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(2);
expect(elInput.selectionEnd).toEqual(3);
});
expect(elInput).toHaveValue("1234");
await userEvent.keyboard("{backspace}{/backspace}");
expect(elInput).toHaveValue("124");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(2);
expect(elInput.selectionEnd).toEqual(3);
});
// Switched to type mode
await userEvent.keyboard("{ArrowRight}{/ArrowRight}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(3);
});
// Can't move further
await userEvent.keyboard("{ArrowRight}{/ArrowRight}");
expect(elInput.selectionStart).toEqual(3);
});
Unfortunately, it still fails. This time I kept track of my tries (out of 10): it failed on the first, third, fifth, sixth.
Summary of all failing tests
FAIL src/components/PinField/tests/PinField.test.tsx (26.09 s)
● Components/PinField › handles keyboard navigation
expect(received).toEqual(expected) // deep equality
Expected: 2
Received: 1
Ignored nodes: comments, script, style
<html
data-rs-color-mode="light"
data-rs-keyboard="true"
data-rs-theme="reshaped"
dir="ltr"
>
<head />
<body>
<div>
<div
class=" root root"
>
<div
class=" root root --position-relative --flex --direction-row --nowrap-false"
style="--rs-view-gap-s: 2;"
>
<div
class=" root item root --radius-small --position-relative unit unit root --border-neutral --bg-elevation-base --flex --direction-column --align-center --justify-center --nowrap-false"
style="--rs-w-s: 9; --rs-h-s: 9;"
>
<div
class=" root --variant-body-3"
>
1
</div>
</div>
<div
class=" root item item--focused root --radius-small --position-relative unit unit root --border-neutral --bg-elevation-base --flex --direction-column --align-center --justify-center --nowrap-false"
style="--rs-w-s: 9; --rs-h-s: 9;"
>
<div
class=" root --variant-body-3"
>
2
</div>
</div>
<div
class=" root item root --radius-small --position-relative unit unit root --border-neutral --bg-elevation-base --flex --direction-column --align-center --justify-center --nowrap-false"
style="--rs-w-s: 9; --rs-h-s: 9;"
>
<div
class=" root --variant-body-3"
>
3
</div>
</div>
<div
class=" root item root --radius-small --position-relative unit unit root --border-neutral --bg-elevation-base --flex --direction-column --align-center --justify-center --nowrap-false"
style="--rs-w-s: 9; --rs-h-s: 9;"
>
<div
class=" root --variant-body-3"
>
4
</div>
</div>
<input
autocomplete="one-time-code"
class="input"
inputmode="numeric"
maxlength="4"
name="test-name"
pattern="\\d{4}"
type="text"
value="1234"
/>
</div>
</div>
</div>
</body>
</html>
149 |
150 | await waitFor(() => {
> 151 | expect(elInput.selectionStart).toEqual(2);
| ^
152 | expect(elInput.selectionEnd).toEqual(3);
153 | });
154 |
at src/components/PinField/tests/PinField.test.tsx:151:35
at runWithExpensiveErrorDiagnosticsDisabled (node_modules/@testing-library/dom/dist/config.js:47:12)
at checkCallback (node_modules/@testing-library/dom/dist/wait-for.js:124:77)
at checkRealTimersCallback (node_modules/@testing-library/dom/dist/wait-for.js:118:16)
at Timeout.task [as _onTimeout] (node_modules/jsdom/lib/jsdom/browser/Window.js:520:19)
Test Suites: 1 failed, 78 passed, 79 total
Tests: 1 failed, 1 todo, 334 passed, 336 total
Snapshots: 72 passed, 72 total
Time: 45.489 s
I also forgot to initially mention that these fails are present on Github actions too, so it's not limited to local. Here is my pull-request.yml action:
name: "pull-request"
on:
pull_request:
branches:
- main
jobs:
test-pull-request:
permissions: write-all
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: "20.x"
- run: yarn
- run: yarn lint
- run: yarn build
- run: yarn test:unit
- uses: andresz1/[email protected]
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
build_script: build
By the way, ChatGPT is suggesting that we try a timeout on the waitFor on the tests like so:
await userEvent.keyboard("{ArrowRight}{/ArrowRight}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(3);
}, { timeout: 5000 }); // Adjust timeout if necessary
Interesting, does it help if you try to add some artificial timeout? Having it failing in github actions too makes me quite confused 😅
I've tried that, global async timeout and a lot of other approaches. I can't manage to get it fixed!
Strangely through, running an isolated test like yarn test:unit src/components/PinField/tests/PinField.test.tsx never fails. Seems like it's failing only when running alongside the other tests.
I got it reproduced once in my CI yesterday so hopefully it keeps failing today 😅
Another attempt, where I try to wait for every single keypress to resolve before moving on. I'm not 100% sure this is the issue but at least it stopped reproducing locally, can you check this one?
test("handles keyboard navigation", async () => {
render(
<Reshaped>
<PinField name={fixtures.name} />
</Reshaped>
);
const elInput = screen.getByRole<HTMLInputElement>("textbox");
await act(() => {
elInput.focus();
});
expect(elInput.selectionStart).toEqual(0);
expect(elInput.selectionEnd).toEqual(0);
await userEvent.keyboard("1");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(1);
expect(elInput.selectionEnd).toEqual(1);
});
await userEvent.keyboard("234");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(3);
expect(elInput.selectionEnd).toEqual(4);
});
// Move back to the first character
await userEvent.keyboard("{ArrowLeft}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(2);
expect(elInput.selectionEnd).toEqual(3);
});
await userEvent.keyboard("{ArrowLeft}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(1);
expect(elInput.selectionEnd).toEqual(2);
});
await userEvent.keyboard("{ArrowLeft}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(0);
expect(elInput.selectionEnd).toEqual(1);
});
// Move to the third character
await userEvent.keyboard("{ArrowRight}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(1);
expect(elInput.selectionEnd).toEqual(2);
});
await userEvent.keyboard("{ArrowRight}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(2);
expect(elInput.selectionEnd).toEqual(3);
});
expect(elInput).toHaveValue("1234");
await userEvent.keyboard("{backspace}");
expect(elInput).toHaveValue("124");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(2);
expect(elInput.selectionEnd).toEqual(3);
});
// Switched to type mode
await userEvent.keyboard("{ArrowRight}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(3);
expect(elInput.selectionStart).toEqual(3);
});
// Can't move further
await userEvent.keyboard("{ArrowRight}");
await waitFor(() => {
expect(elInput.selectionStart).toEqual(3);
expect(elInput.selectionStart).toEqual(3);
});
});
I've ran it ~20 times and none of them failed! Nice one, that's the solution!
You've probably seen this, but just dropping this warning during the tests:
RUNS src/components/Calendar/tests/Calendar.test.tsx
● Console
console.error
Warning: `ReactDOMTestUtils.act` is deprecated in favor of `React.act`. Import `act` from `react` instead of `react-dom/test-utils`. See https://react.dev/warnings/react-dom-test-utils for more info.
at Component (/Users/vladoyoung/Downloads/design-system/src/hooks/tests/useRTL.test.tsx:8:30)
at ToastProvider (/Users/vladoyoung/Downloads/design-system/src/components/Toast/ToastProvider.tsx:63:10)
at SingletonHotkeysProvider (/Users/vladoyoung/Downloads/design-system/src/hooks/_private/useSingletonHotkeys.tsx:157:10)
at ReshapedInner (/Users/vladoyoung/Downloads/design-system/src/components/Reshaped/Reshaped.tsx:17:10)
at div
at Theme (/Users/vladoyoung/Downloads/design-system/src/components/Theme/Theme.tsx:12:10)
at GlobalColorMode (/Users/vladoyoung/Downloads/design-system/src/components/Theme/GlobalColorMode.tsx:10:10)
at Reshaped (/Users/vladoyoung/Downloads/design-system/src/components/Reshaped/Reshaped.tsx:32:10)
9 |
10 | React.useEffect(() => {
> 11 | act(() => setRTL(true));
| ^
12 | }, [setRTL]);
13 |
14 | return <div>{rtl}</div>;
at printWarning (node_modules/react-dom/cjs/react-dom-test-utils.development.js:71:30)
at error (node_modules/react-dom/cjs/react-dom-test-utils.development.js:45:7)
at actWithWarning (node_modules/react-dom/cjs/react-dom-test-utils.development.js:1736:7)
at src/hooks/tests/useRTL.test.tsx:11:6
at commitHookEffectListMount (node_modules/react-dom/cjs/react-dom.development.js:23189:26)
at commitPassiveMountOnFiber (node_modules/react-dom/cjs/react-dom.development.js:24970:11)
at commitPassiveMountEffects_complete (node_modules/react-dom/cjs/react-dom.development.js:24930:9)
at commitPassiveMountEffects_begin (node_modules/react-dom/cjs/react-dom.development.js:24917:7)
at commitPassiveMountEffects (node_modules/react-dom/cjs/react-dom.development.js:24905:3)
at flushPassiveEffectsImpl (node_modules/react-dom/cjs/react-dom.development.js:27078:3)
at flushPassiveEffects (node_modules/react-dom/cjs/react-dom.development.js:27023:14)
at performSyncWorkOnRoot (node_modules/react-dom/cjs/react-dom.development.js:26115:3)
at flushSyncCallbacks (node_modules/react-dom/cjs/react-dom.development.js:12042:22)
at commitRootImpl (node_modules/react-dom/cjs/react-dom.development.js:26998:3)
at commitRoot (node_modules/react-dom/cjs/react-dom.development.js:26721:5)
at finishConcurrentRender (node_modules/react-dom/cjs/react-dom.development.js:26020:9)
at performConcurrentWorkOnRoot (node_modules/react-dom/cjs/react-dom.development.js:25848:7)
at flushActQueue (node_modules/react/cjs/react.development.js:2667:24)
at act (node_modules/react/cjs/react.development.js:2582:11)
at node_modules/@testing-library/react/dist/act-compat.js:47:25
at renderRoot (node_modules/@testing-library/react/dist/pure.js:180:26)
at render (node_modules/@testing-library/react/dist/pure.js:271:10)
at Object.<anonymous> (src/hooks/tests/useRTL.test.tsx:19:9)