Problem with nuxt imports: Vitest Coverage stays at 100% for components regardless of the existence of any component test because of nuxt imports

[FlorianPhilipp2007]

Environment

• Operating System: Darwin
• Node Version: v18.16.0
• Nuxt Version: 3.11.2
• CLI Version: 3.11.1
• Nitro Version: 2.9.6
• Package Manager: [email protected]
• Builder: -
• User Config: devtools, typescript, extends, components, ssr, runtimeConfig, app, css, build, modules, experimental, i18n, proxy, vite, public, nitro, hooks
• Runtime Modules: @pinia/[email protected], [email protected], [email protected], @nuxt/test-utils/[email protected], @nuxtjs/[email protected]
• Build Modules: -

Reproduction

https://stackblitz.com/edit/nuxt-starter-gvfsue?file=nuxt.config.ts,components%2Fnested%2FTestNestedUntested.vue

vitest --coverage is not working in StackBlitz. Please download the project and try it locally.

Describe the bug

Description: When the components option in the Nuxt configuration is set to true (enabling automatic global registration of components), the test coverage report shows 100% coverage for all components, regardless of whether tests actually exist for them. Something is importing the untested TestNestedUntested. V8 is reporting coverage for it. Maybe Nuxt imports all those files.

Actual Behavior: The coverage report incorrectly shows 100% coverage for all components, including those without any tests like TestNestedUntested.vue in component directory. Seems to be a problem caused by nuxt importing all files.

Expected Behavior: The coverage report should accurately reflect the coverage for each component. Only TestNested.vue should show 100% coverage, while TestNestedUnested.vue should show 0% coverage or appropriate coverage based on the absence of tests.

Additional context

No response

Logs

No response

[bolt-new-by-stackblitz[bot]]

Start a new pull request in StackBlitz Codeflow.

• nuxt-starter-gvfsue (Open reproduction)

[danielroe]

This is unlikely something to fix in Nuxt. It might possibly be a vitest bug or something to be resolved in nuxt/test-utils. Have you tried ensuring that .nuxt is added to your coverage exclude patterns?

[dosubot[bot]]

Hey @FlorianPhilipp2007! I'm here to help you with any bugs, questions, or contributions. Let's work together to find a solution!

To resolve the issue where Nuxt imports are causing Vitest coverage reports to incorrectly show 100% coverage for all components, you need to adjust the coverage configuration in your vitest.config.ts file. Specifically, you should ensure that the coverage configuration accurately reflects the files and directories you want to include and exclude from the coverage report.

Here's an updated version of your vitest.config.ts file with a more precise coverage configuration:

import { resolve } from 'pathe'
import { configDefaults, coverageConfigDefaults, defineConfig } from 'vitest/config'
import { isWindows } from 'std-env'

export default defineConfig({
  resolve: {
    alias: {
      '#build/nuxt.config.mjs': resolve('./test/mocks/nuxt-config'),
      '#internal/nuxt/paths': resolve('./test/mocks/paths'),
      '#build/app.config.mjs': resolve('./test/mocks/app-config'),
      '#app': resolve('./packages/nuxt/dist/app'),
    },
  },
  test: {
    globalSetup: './test/setup.ts',
    setupFiles: ['./test/setup-env.ts'],
    coverage: {
      exclude: [...coverageConfigDefaults.exclude, 'packages/nuxt/src/app', 'playground', '**/test/', 'scripts', 'vitest.nuxt.config.ts'],
    },
    testTimeout: isWindows ? 60000 : 10000,
    exclude: [...configDefaults.exclude, 'nuxt/**', '**/test.ts', '**/this-should-not-load.spec.js'],
    poolOptions: {
      threads: {
        maxThreads: process.env.TEST_ENV === 'dev' ? 1 : undefined,
        minThreads: process.env.TEST_ENV === 'dev' ? 1 : undefined,
      },
    },
  },
})

In this configuration:

• The resolve.alias option is used to mock certain Nuxt paths, ensuring that the correct files are referenced during testing.
• The coverage.exclude option is set to exclude specific directories and files from the coverage report, including the packages/nuxt/src/app directory and test-related files.
• The test.exclude option is set to exclude certain files and directories from being tested, ensuring that only the relevant files are included in the coverage report.

This should help ensure that the coverage report accurately reflects the tested and untested components ^[1][2].

To continue talking to Dosu, mention @dosu.

[FlorianPhilipp2007]

This is unlikely something to fix in Nuxt. It might possibly be a vitest bug or something to be resolved in nuxt/test-utils. Have you tried ensuring that .nuxt is added to your coverage exclude patterns?

I first raised the issue at vitest but they think its a nuxt problem: https://github.com/vitest-dev/vitest/issues/6029#issuecomment-2210914420

I tried the exclude from .nuxt but it doesen't help

[danielroe]

It seems like the issue is the one Ari linked - it's a Vue sourcemap bug: https://github.com/vitest-dev/vitest/issues/3607#issuecomment-1608954728.

[AriPerkkio]

This report contains two issues:

1. Something (nuxt ?) is importing files that are not being tested. Node's V8 coverage includes every file that is loaded via Node. The repro is testing TestNested.vue, but is also unexpectedly seeing coverage for UntestNested.vue. Vitest is not importing UntestNested.vue.
2. Coverage is incorrectly 100%. This is caused by Vue's sourcemaps being off. We've seen so many of these issues that I think I'll need to raise an issue on Vue some day.

[DenisLug]

@AriPerkkio, @danielroe: Thank you both for your help so far.

The issue with the coverage report has evolved after updating vitest to version 2 (see comment in vitest issue, Stackblitz demo). Previously, the coverage incorrectly showed 100% for all components, but now it shows 0% coverage for files that do have tests.

To summarize the current status:

• With the istanbul provider, the coverage report collects only the coverage for the utils directory
• With the v8 provider, the coverage report now shows 0% coverage for files that have tests

Given these changes, could we clarify who will be investigating this further? Is this something that should be addressed by the Nuxt team, or should we continue the discussion with the Vitest team?

[DenisLug]

I have retested the issue with the new vitest major version 3 and the issue still persists. See here for the updated reproduction: https://stackblitz.com/edit/github-hfxk3ssz

[ademyalcin27]

Is there any update with the latest(vitest 3.0.8) versions?

[ulysse-lacour]

Having the same problem here with 3.0.8, all files are to 0 despite being tested

[jsedanoj]

https://github.com/nuxt/nuxt/issues/31384 has been closed as duplicate of this one, so I will share the diagnosis here (see the link for more detail, instructions, screenshots and reproduction in a repo, including the workaround).

Starting from the basic Nuxt application as bootstrapped from the documentation, when I add .vue files to the pages/ directory, the coverage is not measured. The other folders work. Renaming the directory to pagesX/ solves the coverage, but breaks the application (obviously). Configuring nuxt.config:dirs.pages to pagesX fixes the application, but breaks the coverage again.

Apparently, Nuxt is doing something with the pages/ directory that breaks the coverage.

[danielroe]

@jsedanoj if you think it's a different issue we can open a new one but it should be in https://github.com/nuxt/test-utils.

[jsedanoj]

Well, you know better than me. If you think the underlying problem is the same, we can keep it here. If you prefer to create a new one for tracking (because the symptoms are different) I can create it, no problem. Just tell me and I will (I have the repo with the reproduction and the text, so it is a matter of minutes). I'll do as you prefer.

[MatteoGabriele]

I'm experiencing a similar issue. I created a small application to identify which package was causing the problem, and it appears to be Vitest v3 when testing a Nuxt 3 project. When I run the same code and tests using Vitest v2, the coverage report is accurate. However, with v3, the components inside the /pages directory are ignored. I’ve attached a simple side-by-side screenshot comparing Vitest v2 and v3. Let me know if you need a reproduction for further investigation; I’d be happy to help.

[jsedanoj]

If you already have the project done, maybe you could push it to a repo, as example of workaround. If it works for me, maybe I can just downgrade my vitest, instead of using my workaround.

This may make us think that the problem is inside vitest, actually. But considering that it only fails for one directory of nuxt... and being this directory "pages/" that we know it has special meaning for nuxt...

[MatteoGabriele]

The test repository contains a component and a page with the same code and tests. The main branch uses Vitest 3 but with the broken code coverage. There is another branch called "vitest2" that has working code coverage, which involves downgrading to Vitest v2 and @vitest/coverage-v8 v2.

I'm uncertain about which library is causing the issue. It appears to be related to Vitest, but since the problem involves components within the pages folder, I suspect that Nuxt might be encountering an issue with something in Vitest v3.

https://github.com/MatteoGabriele/nuxt-vitest-project

[ademyalcin27]

I have an example here , I commented some of test, it show 100%. https://stackblitz.com/edit/nuxt-starter-gjkmvknv?file=pages%2Findex.vue

Another interesting thing is I have the same configuration in my local but I got different results as below. and sometimes it goes endless loop, so the test is rerun until I stop

[MatteoGabriele]

just wanted to point out that similar issues with code coverage and Vitest v3 have been spotted in the i18n module https://github.com/nuxt-modules/i18n/issues/3238#issuecomment-2672492536 Maybe it helps, maybe it doesn't. In my case, it did help a little since I now have coverage of all components, but the pages are still at 0.

[jsedanoj]

And now it is happening for me also in other folders on the project (.vue files in components/). So far, I have found no workaround, but it is systematic. I will try to add this to my minimal example, if I can simplify it.

In my search through the internet some people says that maybe the sourcemap is incorrect... well, it makes sense. But I do not have enough experience with the sourcemap to say if it is correct or not.

[MatteoGabriele]

To eliminate any uncertainties, I have made the exact same application using plain Vue, VueRouter, and Vitest, and it works as expected without any issues. I had to try it.

[MatteoGabriele]

The coverage will work if I change the folder name of "pages" to "views", for example. It's important to note that changing the value of dir.pages in the nuxt.config file from the default "pages" to something like "views" will cause the coverage to fail again.

[ineshbose]

From my research, seems that the thread to follow is: https://github.com/vitest-dev/vitest/issues/4993

[jsedanoj]

@MatteoGabriele , did you create a bug report in vitest for this?

[MatteoGabriele]

@jsedanoj not really. I thought that this was a nuxt error since it is so specific about pages.

[jsedanoj]

Actually... now for me it fails for components/ as well. https://github.com/nuxt/test-utils/issues/891#issuecomment-2750270671

Really, I cannot believe that obtaining a proper code coverage report is that hard. I tried changing to istambul, no luck. I tried going back to vitest2... good luck with pages/, but not with components/.

This is a sooooo basic concept in software engineering, that I expected this to work out-of-the-box just by following the Getting Started guide.

Does anybody have a working example of a Nuxt3 project with a proper testing and coverage?

[MatteoGabriele]

I completely get the frustration, but just a reminder that open source relies on community effort. It’s important to stay constructive and supportive so we can all work together to improve things.

If your previous setup worked better, it might be worth sticking with that for now. Personally, I'm using the latest Nuxt 3 with Vitest 2 and haven’t run into issues. You can also check my reproduction where the "vitest2" branch works as expected.

Most importantly, let’s not forget to appreciate the hard work that goes into these projects. Gratitude really goes a long way.

[jsedanoj]

Yes, you are right, the frustration was speaking.

The problem for one of those "previous setups" is that... it was Nuxt2, out of support now. So not an option anymore.

[MatteoGabriele]

@jsedanoj, no worries! If you are using Nuxt 2, you must have had a moment before the recent updates when you were able to achieve the correct coverage, right? Have you tried using an exact version of Vitest 2? It may be a tedious job, but consider reverting to several earlier minor versions until you find one that works.

[jsedanoj]

Not really.

In one of the projects, I was migrating from Nuxt2 to Nuxt3, and I thought it worked because my tests where giving 100% coverage... until I have discovered, during this investigation, that it actually didn't, it just always says 100% with v8, and completely non-sensical lines with istambul.

In another project, I was moving from another architecture to Nuxt3 directly... so yes, I will just stay in the other architecture (express + pure Vue3), it works and is well maintained.

So it is not because of lack of options.

[AriPerkkio]

I have 0 experience of Nuxt, but the minimal reproduction seems to work when setting components.global to false in nuxt.config.ts.

When it's set to true, coverage report is odd:

What exactly does components.global do? How should testing tools handle it?