testing-library
`getByRole` fails when running under `vitest` with `--no-isolate --no-file-parallelism`
-
@testing-library/domversion: 10.4.0 - Testing Framework and version: vitest 2.1.2
- DOM Environment: jsdom 25.0.1
Relevant code or config:
Have two files with following content
// ./foo.test.ts
import { screen } from '@testing-library/dom';
describe('foo suite', () => {
it('foo', async () => {
console.log('foo');
const button = document.createElement('button');
document.body.appendChild(button);
try {
screen.debug();
screen.getByRole('button');
} finally {
button.remove();
}
});
});
// ./bar.test.ts
import { screen } from '@testing-library/dom';
describe('bar suite', () => {
it('bar', async () => {
console.log('bar');
const button = document.createElement('button');
document.body.appendChild(button);
try {
screen.debug();
screen.getByRole('button');
} finally {
button.remove();
}
});
});
What you did:
Run pnpm test run => works
Run pnpm test run -- --no-isolate --no-file-parallelism => one of the tests fail.
What happened:
One of the tests fails, it can't seem to locate the button, yet it's there when calling screen.debug()
bar
<body>
<button />
</body>
/home/projects/vitest-dev-vitest-u1eg7t/test/suite.test.tsx:10:14
8 | document.body.appendChild(button);
9 | try {
> 10 | screen.debug();
| ^
stdout | test/basic.test.tsx > foo suite > foo
foo
<body>
<button />
</body>
/home/projects/vitest-dev-vitest-u1eg7t/test/basic.test.tsx:10:14
8 | document.body.appendChild(button);
9 | try {
> 10 | screen.debug();
| ^
❯ test/basic.test.tsx (1)
❯ foo suite (1)
× foo
✓ test/suite.test.tsx (1)
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
FAIL test/basic.test.tsx > foo suite > foo
TestingLibraryElementError: Unable to find an accessible element with the role "button"
There are no accessible roles. But there might be some inaccessible roles. If you wish to access them, then set the `hidden` option to `true`. Learn more about this here: https://testing-library.com/docs/dom-testing-library/api-queries#byrole
Ignored nodes: comments, script, style
Reproduction:
Stackblitz with reproduction: https://stackblitz.com/edit/vitest-dev-vitest-9xcyxz?file=test%2Fbar.test.ts
You can run pnpm test run and pnpm test run -- --no-isolate --no-file-parallelism to see the bug in action.
Problem description:
Migrating from mocha to vitest, trying to reach similar performance by disabling isolation
Suggested solution:
edit: hidden because it' not caused by @testing-library/react
Ok, got a little more insight into how vitest handles these modes. Not sure I agree with the design decisions taken there, given that it's a "won't fix", I suppose there isn't much to be done on this end.
Hi @Janpot, thanks for opening this.
I'm not sure there's anything we can do here.
This isn't tightly related to testing library as the --no-isolate flag clearly states that this option is usually good to use in node environment but the environment you're using here is JSDOM that means that you must have a clear DOM before every run starts.
Also, the reproduction actually passed for me:
Which only states that this flakiness is related to the isolation issue.
I honestly don't think there's an issue with testing library here or anything we can do to make this work better for your use case. I'm closing this as not planned, if you disagree, please feel free to comment and re-open.
Uh oh! @MatanBobi, the image you shared is missing helpful alt text. Check https://github.com/testing-library/dom-testing-library/issues/1337#issuecomment-2428499737.
Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.
Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
edit: ignore this comment, I didn't look deep enough into this. This seems to be fixed in latest vitest.
@MatanBobi Yes, the default command shows the working version, to reproduce the issue, one needs to run pnpm test run -- --no-isolate --no-file-parallelism manually. I've changed the default test command in the reproduction so that it fails now. It's not flakyness, it fails consistently.
that means that you must have a clear DOM before
The tests clean up the dom after themselves. The prerequisites are therefore fulfilled. Are you saying that merely running a jsdom test without even modifying the DOM pollutes the test env so much that it needs a full reset? That sure can't be by design?
please feel free to comment and re-open.
I'm not able to, but kindly request to reopen.
if you disagree, please feel free to comment and re-open.
@MatanBobi I'm in the process of migrating our 500+ suites from mocha to vitest. I've noticed a huge slowdown. around 3x slower to run our tests in vitest. We can emulate the mocha way of running tests by running vitest with --no-isolate --no-file-parallelism. This causes the vitest to run similarly to mocha, each test sequentially, all in one process. This brings our runtime back to be very similar to mocha. The main difference though is that vitest jsdom environment still recreates a dom for each suite. Consequently there will be new document and window globals for each suite. It looks like global screen uses the initial document and window and assumes they stay constant through all tests. All our tests run perfectly when we run them with local screen (const screen = render(...)). I feel like a 3x slowdown is a serious regression compared to mocha and the fix seems to be quite straightforward, don't assume document never gets reassigned. (At least the fix was straightforward in user-event). For that reason I propose to reopen.
edit: fyi, also opened https://github.com/vitest-dev/vitest/issues/8478 in the meantime.