vitest-dev
Incorrect test results on *first* run after installing node_modules
Describe the bug
Using browser mode and either v8 or istanbul coverage, the test run fails the first time after installing node_modules with error Failed to fetch dynamically imported module: http://localhost:5173/@id/@vitest/coverage-v8/browser?import. This is currently blocking me from upgrading because tests fail 100% of the time in CI since it's a fresh node_modules.
> coverage
> vitest run --coverage
RUN v2.1.0-beta.6 /Users/kevinwojcik/ontra/eng/vitestsourcemapbug
Browser runner started at http://localhost:5173/
Coverage enabled with v8
5:07:40 PM [vite] ✨ new dependencies optimized: @vitest/coverage-v8/browser
5:07:40 PM [vite] ✨ optimized dependencies changed. reloading
5:07:41 PM [vite] ✨ new dependencies optimized: react/jsx-dev-runtime
5:07:41 PM [vite] ✨ optimized dependencies changed. reloading
stderr | test/basic.test.tsx > Link changes the state when hovered
Warning: `ReactDOMTestUtils.act` is deprecated in favor of `React.act`. Import `act` from `react` instead of `react-dom/test-utils`. See https://react.dev/warnings/react-dom-test-utils for more info.
✓ test/basic.test.tsx (1)
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Rejection ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
TypeError: Failed to fetch dynamically imported module: http://localhost:5173/@id/@vitest/coverage-v8/browser?import
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Test Files 1 passed (1)
Tests 1 passed (1)
Errors 1 error
Start at 17:07:39
Duration 2.11s (transform 0ms, setup 125ms, collect 213ms, tests 13ms, environment 0ms, prepare 45ms)
% Coverage report from v8
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 0 | 0 | 0 | 0 |
Link.tsx | 0 | 0 | 0 | 0 | 1-32
----------|---------|----------|---------|---------|-------------------
AFAICT, this was introduced in 2.1.0-beta.1. It works fine in 2.0.5, but look at how basic.test.tsx gets run twice (see below)
> coverage
> vitest run --coverage
RUN v2.0.5 /Users/kevinwojcik/ontra/eng/vitestsourcemapbug
Browser runner started at http://localhost:5173/
Coverage enabled with istanbul
5:11:22 PM [vite] ✨ new dependencies optimized: react/jsx-dev-runtime
5:11:22 PM [vite] ✨ optimized dependencies changed. reloading
stderr | test/basic.test.tsx > Link changes the state when hovered
Warning: `ReactDOMTestUtils.act` is deprecated in favor of `React.act`. Import `act` from `react` instead of `react-dom/test-utils`. See https://react.dev/warnings/react-dom-test-utils for more info.
✓ test/basic.test.tsx (1)
5:11:23 PM [vite] ✨ new dependencies optimized: @vitest/coverage-istanbul
stderr | test/basic.test.tsx > Link changes the state when hovered
Warning: `ReactDOMTestUtils.act` is deprecated in favor of `React.act`. Import `act` from `react` instead of `react-dom/test-utils`. See https://react.dev/warnings/react-dom-test-utils for more info.
✓ test/basic.test.tsx (1)
✓ test/foo.test.ts (1000)
Test Files 2 passed (2)
Tests 1001 passed (1001)
Start at 17:11:21
Duration 2.56s (transform 0ms, setup 346ms, collect 45ms, tests 38ms, environment 0ms, prepare 92ms)
% Coverage report from istanbul
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 100 | 50 | 100 | 100 |
Link.tsx | 100 | 50 | 100 | 100 | 22
----------|---------|----------|---------|---------|-------------------
Reproduction
branch: https://github.com/kwojcik/vitestsourcemapbug/tree/coverageCrashRepro
git clone [email protected]:kwojcik/vitestsourcemapbug.git
cd vitestsourcemapbug
rm -r node_modules; npm install --force && npm run coverage
System Info
System:
OS: macOS 13.6.7
CPU: (12) arm64 Apple M2 Max
Memory: 10.74 GB / 64.00 GB
Shell: 5.9 - /bin/zsh
Binaries:
Node: 18.20.4 - ~/.asdf/installs/nodejs/18.20.4/bin/node
Yarn: 1.22.21 - /opt/homebrew/bin/yarn
npm: 10.7.0 - ~/.asdf/plugins/nodejs/shims/npm
pnpm: 9.5.0 - /opt/homebrew/bin/pnpm
Watchman: 2024.08.12.00 - /opt/homebrew/bin/watchman
Browsers:
Chrome: 127.0.6533.122
Firefox Nightly: 127.0
Safari: 16.6
npmPackages:
@vitest/browser: ^v2.1.0-beta.6 => 2.1.0-beta.6
@vitest/coverage-istanbul: ^v2.1.0-beta.6 => 2.1.0-beta.6
@vitest/ui: ^v2.1.0-beta.6 => 2.1.0-beta.6
vite: latest => 5.3.2
vitest: ^v2.1.0-beta.6 => 2.1.0-beta.6
### Used Package Manager
npm
### Validations
- [X] Follow our [Code of Conduct](https://github.com/vitest-dev/vitest/blob/main/CODE_OF_CONDUCT.md)
- [X] Read the [Contributing Guidelines](https://github.com/vitest-dev/vitest/blob/main/CONTRIBUTING.md).
- [X] Read the [docs](https://vitest.dev/guide/).
- [X] Check that there isn't [already an issue](https://github.com/vitest-dev/vitest/issues) that reports the same bug to avoid creating a duplicate.
- [X] Check that this is a concrete bug. For Q&A open a [GitHub Discussion](https://github.com/vitest-dev/vitest/discussions) or join our [Discord Chat Server](https://chat.vitest.dev).
- [X] The provided reproduction is a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) of the bug.
There should be no new dependencies optimized logs at all for Vitest (or any Vite-based testing framework) to work correctly. Having @vitest ones is a bug in Vitest. The react one is interesting, maybe we should add well-known dependencies ourselves.
For now, you can add them to your config manually:
export default defineConfig({
optimizeDeps: {
include: ['react/jsx-dev-runtime'],
exclude: ['@vitest/coverage-istanbul'],
}
})
I guess this doesn't show up in our own tests or ecosystem ci due to locally linked packages. It would be great to be able to replicate this issue in our tests at least. 🤔
@httpete maybe this is what you were experiencing in https://github.com/vitest-dev/vitest/issues/5615?
The @vitest packages should be fixed by #6445
We also now print the warning that Vitest doesn't expect dependencies to be optimized.
Regarding react - please, use @vitejs/plugin-react if you use react in your project. It adds the jsx-transform to include automatically.