Code Coverage Issue with Istanbul Provider

[aimad-majdou]

trafficstars

Describe the bug

I am experiencing an issue with code coverage reporting when using the Istanbul provider. Some files that are exported as default are not marked as covered in the coverage report, even though I have tests for them.

Interestingly, when I add an additional export to these files, they are marked as covered:

Notably, this issue does not occur when I switch to using the V8 provider for code coverage. This behavior seems inconsistent, as other files with only a default export do not require an additional export to get covered.

Reproduction

Check this example: https://stackblitz.com/edit/github-w9fjy7-fjohui

As you can see in image below after the test runs in this example, the file covered.tsx gets covered, but the file notCovered doesn't get covered.

The files and their tests are exactly the same, the only difference is that NotCovered is only exported by a default export, but Covered is exported directly and also by a default export.

Also you can see that the file SubFeature like NotCovered is only exported by a default export, but it gets covered.

System Info

System:
    OS: Windows 10 10.0.19045
    CPU: (8) x64 Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz
    Memory: 4.46 GB / 15.86 GB
  Binaries:
    Node: 18.12.1 - C:\Program Files\nodejs\node.EXE     
    npm: 8.19.2 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.19041.1266.0), Chromium (117.0.2045.41)
  npmPackages:
    @vitejs/plugin-react: 4.0.3 => 4.0.3
    @vitejs/plugin-react-swc: 3.3.2 => 3.3.2
    @vitest/coverage-istanbul: 0.34.2 => 0.34.2
    @vitest/ui: 0.34.1 => 0.34.1
    vite: 4.4.5 => 4.4.5
    vitest: 0.34.3 => 0.34.3

Used Package Manager

npm

Validations

• [X] Follow our Code of Conduct
• [X] Read the Contributing Guidelines.
• [X] Read the docs.
• [X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• [X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• [X] The provided reproduction is a minimal reproducible example of the bug.

[sheremet-va]

I am unable to provide a specific reproduction case for this issue due to its inconsistency and complexity.

Sorry, but do you expect maintainers to come up with reproductions that are inconsistent and complex? It would be nice if you could put your examples in a small reproduction.

[github-actions[bot]]

Hello @aimad-majdou. Please provide a minimal reproduction using a GitHub repository or StackBlitz. Issues marked with need reproduction will be closed if they have no activity within 3 days.

[aimad-majdou]

@sheremet-va Thank you for your feedback and understanding. I absolutely appreciate the importance of providing a minimal, reproducible example.

However, my initial report was aimed at determining whether this issue with default exports in the context of code coverage is a known one within the vitest team or the community. This was an attempt to gauge whether others have encountered a similar behavior, as it could indicate a broader problem or a configuration issue.

I'll work on creating one to ensure that we can pinpoint the issue more precisely and facilitate its resolution. I'll update the issue with the reproduction once it's ready.

Thank you for your patience and guidance.

[bolt-new-by-stackblitz[bot]]

Start a new pull request in StackBlitz Codeflow.

• github-w9fjy7-fjohui (Open reproduction)

[aimad-majdou]

@sheremet-va please check my post edit, I have provided an example of the issue.

[sheremet-va]

Thank you for your collaboration ❤️

[AriPerkkio]

@aimad-majdou could you try following and see if it fixes the issue for you:

npm i -D @vitejs/plugin-react

vite.config.ts:

+import react from '@vitejs/plugin-react';
-import react from '@vitejs/plugin-react-swc';

Or add an unused export to notCovered.tsx exactly as covered.tsx has:

-const NotCovered = ({
+export const NotCovered = ({

Still need to figure out why this is changing the results though.

[AriPerkkio]

Here is minimal reproduction setup without Vitest: https://github.com/AriPerkkio/swc-istanbul-reproduction/blob/main/debug-swc-sourcemaps.mjs

When @swc/core's jsx.transform.react.runtime option is set to 'automatic', the file is marked as uncovered:

-----------|---------|----------|---------|---------|-------------------
File       | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
-----------|---------|----------|---------|---------|-------------------
All files  |       0 |        0 |       0 |       0 |                   
 repro.tsx |       0 |        0 |       0 |       0 |                   
-----------|---------|----------|---------|---------|-------------------

When it's set to classic the file is marked as covered:

-----------|---------|----------|---------|---------|-------------------
File       | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
-----------|---------|----------|---------|---------|-------------------
All files  |     100 |       60 |     100 |     100 |                   
 repro.tsx |     100 |       60 |     100 |     100 | 5-7               
-----------|---------|----------|---------|---------|-------------------

The source maps seem fine but for some reason the istanbul-lib-source-maps is unable to trace sources using them. Debugging this is really time consuming but if anyone is interested, look into these lines: https://github.com/AriPerkkio/swc-istanbul-reproduction/blob/3d58aa24b496143442b45217b7b8f1c4f9ee5312/debug-swc-sourcemaps.mjs#L99-L103

Equivalent ones on Vitest:

https://github.com/vitest-dev/vitest/blob/db1ff438075d34389de5bb4f5cd08370a08337d5/packages/coverage-istanbul/src/provider.ts#L132-L133

My guess is that root cause is somewhere in the source-map package.

[garyb01]

I am seeing a similar issue using istanbul.

After adding an extra export:

[bruce-c-liu]

I also have this issue. To avoid having to add an extra, unused export, I was able to resolve it by using a const export. Quite strange.

export const ProfilePage = ()=>{};

[Alletkla]

Same here with v8 and istanbul. @sheremet-va @AriPerkkio

I think its related to the export of functions, cause my default object exports are covered fine:

Having only a default function export, the imported export line is in coverage, but the content of the export isn't, even though called.

Adding unused export as suggested by @bruce-c-liu does not help in my case:

Also defining early and exporting later doesnt help (as implied here: instanbuljs Issue)

Maybe this issues from jest related:

• https://github.com/istanbuljs/istanbuljs/issues/157 ?
• https://github.com/istanbuljs/istanbuljs/issues/162

[AriPerkkio]

@Alletkla unless you are using @vitejs/plugin-react-swc for that express application, you are likely not running into this issue. I would recommend to open new bug report with minimal reproduction. Once @vitejs/plugin-react-swc starts to produce better source maps than the ones below, we'll be closing this issue.

• covered.tsx from the repro
• notCovered.tsx from the repro

[bruce-c-liu]

@Alletkla Several issues with your comment. 😅

1. I wasn't the one who suggested to add an unused export. That was @garyb01.
2. Although it wasn't explicitly mentioned, @garyb01's fix involved using an unused export and an export function ProfilePage(){...}. I believe the export function syntax is also important. At least it worked when I did it exactly like that.
3. The alternative workaround I posted is export const ProfilePage = ()=>{};. This time, it's using an arrow function. That part is also important. (Note that my workaround doesn't involve adding an unused export)

Could you try @garyb01's or my workaround exactly?

[Alletkla]

Sry for the inconvenience @bruce-c-liu

I use no @vitejs/plugin-react-swc still facing the issue.

2.: I tested it with this:

export const asd = 123;

export default function(router: Router) {
  router.use('/auth', route); 

-> No coverage on the export

3. Also tried with arrow function.

export const Test = () => {};

export default function(router: Router) {
 router.use('/auth', route);

//

export const authRoute = (router: Router) => {
  router.use('/auth', route);

But no coverage reported.

Anyways: Thanks for your help :)

I will follow the hint from Ari and create a new issue with an minimal code example, if i can reproduce it outside my project. Maybe on sunday.

[garyb01]

@bruce-c-liu exporting React components using arrow functions does fix the issue for me as well

It's hard to reproduce this issue because it does not happen everywhere and is inconsistent in how it does it. I was not able to create a small test repo that reproduced this issue :(

I hope someone can :)