vitest icon indicating copy to clipboard operation
vitest copied to clipboard

Allow project filtered test runs to report coverage of only filtered projects

Open stramel opened this issue 1 year ago • 4 comments

Clear and concise description of the problem

Currently running vitest run --coverage --project <project_a> will result in all other projects reporting no coverage.

Suggested solution

Report only the coverage of the filtered projects when using the --project flag.

Alternative

By default, set the all flag to false if using --project (not quite the same but close)

Additional context

Getting accurate project coverage while testing in a monorepo is hard and you have to resort to npm run cover -w <project_a> which generates a coverage folder that isn't ignored by default at the project level.

Closest I have gotten is using the all:false flag which still doesn't quite do what I'm looking for

Validations

stramel avatar Aug 13 '24 05:08 stramel

Could you set up a minimal setup that demonstrates the issue? And also write down what is the expected/better result that the setup should produce? I could then use that setup to try different approaches for this.

I wonder if we should resolve config.root here from ctx.projects[].root when --project is specified.

https://github.com/vitest-dev/vitest/blob/85fb94a3081558b01f47bd763ccdaeb5df1b98cb/packages/coverage-v8/src/provider.ts#L372-L375

Though sometimes workspace projects share the same root as they can be just different set of options to test same tests, for example https://github.com/AriPerkkio/aria-live-capture/blob/master/vitest.workspace.ts. But in that case narrowing down the config.root wouldn't break anything.

AriPerkkio avatar Aug 14 '24 05:08 AriPerkkio

Could you set up a minimal setup that demonstrates the issue?

@AriPerkkio Here is a minimal reproduction. ~The coverage command never seems to finish on StackBlitz, but you can download it to run it locally.~ Fixed

And also write down what is the expected/better result that the setup should produce?

Coverage is reported for projects outside of the specified project filter. For example, given a monorepo with two projects, p1 and p2, vitest run --coverage --project=p1 reports coverage for code from p2 at 0% because no tests from p2 were run. I would expect files from p2 to be excluded from coverage when using the project filter.

gtbuchanan avatar Apr 23 '25 22:04 gtbuchanan

Thanks for the repro @gtbuchanan. In StackBlitz you can use coverage.provider: 'istanbul' to get coverage working.

I think we can proceed with adding project.root filtering mentioned on https://github.com/vitest-dev/vitest/issues/6331#issuecomment-2287887422.

AriPerkkio avatar Apr 24 '25 04:04 AriPerkkio

This would be beneficial for honojs/middleware

We're using Codecov Carryforward Flags to report coverage across the monorepo, but the coverage report for a single project includes all the other projects having no coverage

BarryThePenguin avatar Jun 19 '25 23:06 BarryThePenguin

I tried this out https://github.com/honojs/middleware/pull/1265 and it looks very promising, thankyou!

BarryThePenguin avatar Jun 26 '25 08:06 BarryThePenguin