vue-jest icon indicating copy to clipboard operation
vue-jest copied to clipboard

SFC Vue files with tests disappear from test coverage reports

Open wopian opened this issue 3 years ago • 42 comments

I have Jest setup to test files with @vue/vue3-jest and it runs all of the Vue tests just fine, however I came across a peculiar issue regarding test coverage - it was going down in our CI dashboard as I was testing more components.

SFC Vue files without a *.spec.js test show up in the Jest coverage summary with zero coverage as expected. After adding a *.spec.js for a component, it will no longer collect or be listed in the coverage summary in our project.

Regular JavaScript files are unaffected and report coverage as expected.

I have a demo repository setup, however it only partially reproduces our issue - @components/Demo.vue is missing, but the two components in @components/demo/ show up fine (which doesn't happen in our project). Removing the @components/Demo.spec.js makes it show up again with zero coverage.

~~Demo: https://github.com/wopian/jest-coverage-demo~~

Demo moved to PR: https://github.com/vuejs/vue-jest/pull/423

With Demo.spec.js, coverage unreported

Screenshot 2021-11-29 at 11 45 43

With Demo.spec.js.old, (zero) coverage reported

Screenshot 2021-11-29 at 11 44 19
vue: ^3.2.23
jest: ^27.3.1
@vue/vue3-jest: ^27.0.0-alpha.4

wopian avatar Nov 29 '21 11:11 wopian

Odd, definitely a bug. Is this something you might be interested in looking into? I'd recommend trying to reproduce it in this repo next, then we can easily debug, test and deploy a fix. You could add a test project here: https://github.com/vuejs/vue-jest/tree/master/e2e/3.x

lmiller1990 avatar Nov 30 '21 00:11 lmiller1990

I've added a more minimal reproduction project there, though I don't know if I will have the free time to investigate further due to work commitments

wopian avatar Dec 03 '21 15:12 wopian

Facing the same here, I'm trying to find out something that could cause this but I not went so far

vue: ^3.2.26
@vue/cli-plugin-unit-jest: ^5.0.0-rc.1
@vue/vue3-jest: ^27.0.0-alpha.4

lucassonsinlima avatar Feb 15 '22 22:02 lucassonsinlima

If anyone wants to take a look at this, it would be great! I'm too busy to look at vue-jest bugs at this point in time.

lmiller1990 avatar Feb 16 '22 02:02 lmiller1990

for those who are interested: we had the same issue in our project and as workaround we returned to

"vue-jest": "^5.0.0-alpha.10",

for now until this issue is fixed.

jonsalvas avatar Mar 30 '22 13:03 jonsalvas

Interesting, we should find out the different between the code in v5 and @vue/vue3-jest@27 - they are very similar, it's just between the two, we moved from semantic versioning to matching the Jest version and a monorepo. The actual code is quite similar, so if someone can track down what broke and port it to @vue/vue3-jest@27, that'd be great.

lmiller1990 avatar Mar 30 '22 21:03 lmiller1990

We had the same issue, switching the reporter to V8 fixed it.

kinoli avatar Apr 22 '22 21:04 kinoli

I'm not entirely certain if it's related, but we're seeing 'weird' things in coverage as well. (vue2 + vue-jest) I've gone back in our pipeline and narrowed it down to this upgrade:

image

Difference in coverage: image

websmurf avatar May 24 '22 10:05 websmurf

Seeing exactly the same with recent version of (vue2 + vue-jest). Vue2 version 4.5.17, Vue-jest version 3.0.7.

adholland avatar May 25 '22 17:05 adholland

After my last comment, we've tried to reconstruct the coverage by downgrading. In the end, we also needed to downgrade @babel/core to version ^7.16.7 in order to make coverage working again.

So we are now in the situation that coverage is working again but with vue-jest@^3.0.7 and @babel/core@^7.16.7 on Jest 28.

websmurf avatar May 27 '22 12:05 websmurf

Why don't you just use V8?

kinoli avatar May 27 '22 13:05 kinoli

Why don't you just use V8?

V8 is incredibly slow on a GitHub Actions runner and runs into OOM errors for large projects

https://github.com/vuejs/vue-jest/pull/423#issuecomment-990940792

wopian avatar May 27 '22 14:05 wopian

Hey,

Is there a date announced for the correction of this bug?

I'm waiting for this fix because on my application I have 100% coverage everywhere and it's really very annoying.

I tried to revert to previous versions and without success.

The update of the lock package following an audit fix was fatal for me

Thx !

AnthonyRuelle avatar Jun 17 '22 11:06 AnthonyRuelle

Can anyone share a working set of npm packages that gives correct coverage for Vue 2? We were previously using [email protected] with [email protected] and upgraded to v28 with @vue/vue2-jest, now our coverage numbers are completely wrong. Jest is showing 1 statement for the entire SFC instead of any methods/computed properties/etc inside the

joshua-farmlink avatar Jun 21 '22 22:06 joshua-farmlink

There's a tip here you can try which should resolve the issues with the latest version: https://github.com/vuejs/vue-cli/issues/7040#issuecomment-1155461372

websmurf avatar Jun 22 '22 07:06 websmurf

The solution that is proposed is to use the coverprovider v8 + jest 27 is that it? For my part, I don't have a vuecli so I don't quite understand the link with the concern we have. Could we have a summary here of what we need to put in version/config to try to solve this problem?

Thanks in advance !

AnthonyRuelle avatar Jun 22 '22 07:06 AnthonyRuelle

Can anyone share a working set of npm packages that gives correct coverage for Vue 2? We were previously using [email protected] with [email protected] and upgraded to v28 with @vue/vue2-jest, now our coverage numbers are completely wrong. Jest is showing 1 statement for the entire SFC instead of any methods/computed properties/etc inside the

You could try vue-jest@4 - old version but it should work with Vue 2 and Jest 26, and have many updates over vue-jest@3.

If the latest versions has speed/OOM memory issues, we should just fix those.

lmiller1990 avatar Jun 23 '22 00:06 lmiller1990

I upgraded to version 4 of vue-jest with jest in version 24.9 and I still have the problem of 100% coverage everywhere It still does not pass on all the methods / computed / etc

how to get good coverage ?

image

AnthonyRuelle avatar Jun 28 '22 10:06 AnthonyRuelle

Question - does Vue CLI ship this w/ code coverage out of the box? Is it a bug here, or bug in the configuration in your project <-> Jest?

lmiller1990 avatar Jun 30 '22 00:06 lmiller1990

I noticed something strange while testing with Vue 3. If the Vue SFC does not import anything itself, the test coverage is reported correctly.

image

If there are any import statements in the SFC (even import {ref} from 'vue'), the file goes missing from test coverage.

image

jest.config.js

module.exports = {
  preset: '@vue/cli-plugin-unit-jest',
  collectCoverageFrom: ['src/**/*.{vue,js}']
}

Does anyone else see the same behavior?

"vue": "3.2.37",
"@vue/cli-plugin-unit-jest": "5.0.6",
"@vue/test-utils": "2.0.0",
"@vue/vue3-jest": "27.0.0", 
"babel-jest": "27.5.1", 
"jest": "27.5.1", 

EDIT: Switching the coverageProvider setting to "v8", as mentioned by someone in an earlier comment resolves the issue, so it's definitely a problem related to babel, I think.

jrtitus avatar Jul 01 '22 03:07 jrtitus

HI !

To unlock future people looking for a solution to this problem (myself included) it is therefore enough to put (temporarily) our versions in [email protected] and [email protected] and to pass the coverageProvider in "v8" is that right?

I will try on my side if it allows me to recover a coherent coverage

AnthonyRuelle avatar Jul 04 '22 11:07 AnthonyRuelle

HI !

To unlock future people looking for a solution to this problem (myself included) it is therefore enough to put (temporarily) our versions in [email protected] and [email protected] and to pass the coverageProvider in "v8" is that right?

I will try on my side if it allows me to recover a coherent coverage

I have similar setup. Passing v8 for the coverageProvider give me the right statements count but still 0/0 of branches and functions count.

vano20 avatar Jul 09 '22 06:07 vano20

Hi, guys. I'm, getting the same error.

I've tried to use v8 to try to fix it, but I got this error: Screen Shot 2022-08-09 at 06 56 15

Does it happen with any else?

 "vue": "^2.6.12",
 "vue-jest": "3.0.7", // I also have tried ^5.0.0-alpha.10 version
 "jest": "26.6.3", // I also have tried 27.* and 28.* versions

Thanks!

tassioFront avatar Aug 09 '22 10:08 tassioFront

You might want @vue/vue2-jest. You want the version that matches your Jest version (v26, v27, etc).

Vue Jest 3 is about 4 years old and is not updated.

lmiller1990 avatar Aug 09 '22 23:08 lmiller1990

Thanks a lot @lmiller1990 !

Even though I keep getting wrong coverage 🙁

Screen Shot 2022-08-10 at 09 27 13 Screen Shot 2022-08-10 at 09 27 19

Just sharing my files

// jest.conifg
 collectCoverage: true,
  testEnvironment: "jsdom",
  testMatch: ["<rootDir>/**/*.(spec).(ts|tsx)"],
  collectCoverageFrom: ["src/**/*.(vue|ts|js)"],
  transform: {
    "^.+\\.ts$": "ts-jest",
    "^.+\\.vue$": "@vue/vue2-jest", // Update to match your installed version
  },
  coverageProvider: "v8",
  coverageReporters: ["json", "lcov", "text-summary", "html"],
//packages
  "@vue/cli-plugin-babel": "~4.5.17",
    "@vue/cli-plugin-eslint": "~4.5.17",
    "@vue/cli-plugin-router": "~4.5.17",
    "@vue/cli-plugin-typescript": "~4.5.17",
    "@vue/test-utils": "^1.0.3",
    "ts-jest": "^27.1.1",
    "ts-loader": "^9.3.1",
    "jest": "27.5.1"
    "@vue/vue2-jest": "27",
    "ts-node": "^10.9.1",
    "typescript": "~3.9.0",
   "babel-core": "^7.0.0-bridge.0",
    "vue-template-compiler": "^2.6.11",
    "@vue/cli-plugin-unit-jest": "~4.5.17",
    "babel-jest": "27.5.1"

tassioFront avatar Aug 10 '22 12:08 tassioFront

I have the same issue. Exists Workaround?

I tried a lot of combinations but not work I have 100% of coverage allways in typescript code in .vue files.

This is my last test:

"vue": "^2.6.11",

"devDependencies": {
    "@4tw/cypress-drag-drop": "^2.2.1",
    "@babel/core": "^7.18.10",
    "@babel/preset-env": "^7.18.10",
    "@mdi/font": "^7.0.96",
    "@playwright/test": "^1.24.2",
    "@types/concat-stream": "^2.0.0",
    "@types/jest": "^28.1.6",
    "@types/js-beautify": "^1.13.3",
    "@types/jsdom": "^20.0.0",
    "@types/lodash.clonedeep": "^4.5.7",
    "@types/mssql": "^8.0.3",
    "@types/node": "^18.6.5",
    "@types/sanitize-html": "^2.6.2",
    "@types/uuid": "^8.3.4",
    "@typescript-eslint/eslint-plugin": "^5.33.0",
    "@typescript-eslint/parser": "^5.33.0",
    "@vue/cli-overlay": "4.5.17",
    "@vue/cli-plugin-babel": "4.5.17",
    "@vue/cli-plugin-eslint": "4.5.17",
    "@vue/cli-plugin-router": "4.5.17",
    "@vue/cli-plugin-typescript": "4.5.17",
    "@vue/cli-plugin-unit-jest": "4.5.17",
    "@vue/cli-plugin-vuex": "4.5.17",
    "@vue/cli-service": "4.5.17",
    "@vue/cli-shared-utils": "4.5.17",
    "@vue/eslint-config-prettier": "^6.0.0",
    "@vue/eslint-config-typescript": "^11.0.0",
    "@vue/test-utils": "^1.3.0",
    "babel-jest": "^24.9.0",
    "concat-stream": "^2.0.0",
    "cypress": "^10.4.0",
    "eslint": "^6.8.0",
    "eslint-plugin-prettier": "^3.4.1",
    "eslint-plugin-tsdoc": "^0.2.16",
    "eslint-plugin-vue": "^9.3.0",
    "jest": "26.6.3",
    "jest-junit": "^14.0.0",
    "jest-sonar-reporter": "^2.0.0",
    "jsdom": "^20.0.0",
    "mssql": "^8.1.3",
    "playwright-chromium": "^1.24.2",
    "prettier": "^2.7.1",
    "sass": "^1.54.0",
    "sass-loader": "^10.2.1",
    "ts-jest": "^26",
    "typescript": "^4",
    "vue-cli-plugin-vuetify": "^2.5.1",
    "vue-jest": "3.0.7",
    "vue-template-compiler": "^2.6.11",
    "vuetify-loader": "^1.7.3"
},

and my jest.config.js:

module.exports = {
  moduleFileExtensions: ["js", "jsx", "json", "vue", "ts", "tsx"],
  transform: {
      "^.+\\.vue$": "vue-jest",
      ".+\\.(css|styl|less|sass|scss|svg|png|jpg|ttf|woff|woff2)$": "jest-transform-stub",
      "^.+\\.tsx?$": "ts-jest",
      ".*\\.(js)$": "babel-jest"
  },
  transformIgnorePatterns: ["/node_modules/"],
  moduleNameMapper: {
      "^@/(.*)$": "<rootDir>/src/$1"
  },
  snapshotSerializers: ["jest-serializer-vue"],
  testMatch: ["**/__tests__/*spec.(js|jsx|ts|tsx)", "!cypress/**/*spec.(js|jsx|ts|tsx)"],
  testPathIgnorePatterns: ["/cypress/"],
  testURL: "http://localhost/",
  watchPlugins: ["jest-watch-typeahead/filename", "jest-watch-typeahead/testname"],
  globals: {
      "ts-jest": {
          babelConfig: true
      }
  },
  coverageProvider: 'v8',
  collectCoverageFrom: ["src/**/*.{ts,vue}", ...ignoreCoverageFrom],
  coverageReporters: ["text", "lcov", "cobertura"],
  reporters: ["default", ["jest-junit", { suiteName: "Unit Tests", outputDirectory: "./junit", outputName: "jest-junit-ut.xml" }]]
}

Thanks!

GrRivero avatar Aug 10 '22 13:08 GrRivero

Silly question but is it possible you are actually getting 100% code coverage? Or are you just importing the component and not doing anything in your test?

lmiller1990 avatar Aug 10 '22 23:08 lmiller1990

Silly question but is it possible you are actually getting 100% code coverage? Or are you just importing the component and not doing anything in your test?

Sorry but it worked before and I don't know which library or libraries has stopped working. The tests work but it does not show coverage or allow real-time debugging in them. 😟

Edited to add new info:

I triyed with a new project and the code coverage doesn´t work

New project: https://github.com/GrRivero/vue2-jest-ts-coverage

New Issue: https://github.com/vuejs/vue-cli/issues/7270

Do you have thoughts on why that is happening this?

Thanks!

GrRivero avatar Aug 11 '22 06:08 GrRivero

I triyed with a new project and the code coverage doesn´t work

@GrRivero, I have done the same and also getting the same strange behavior. :/

tassioFront avatar Aug 11 '22 13:08 tassioFront

Any update on it?

schirrel avatar Oct 13 '22 19:10 schirrel