nyc
nyc copied to clipboard
Code coverage with "all" option does not report uncovered source code
Link to bug demonstration repository
https://github.com/toaditoad/code-coverage-mwe
Expected Behavior
Code coverage should be less than 100% because there is only one test spec FooPage.spec.js
covering the source code of FooPage.vue
. BarPage.vue
should be reported as uncovered.
Observed Behavior
The code coverage is 100%. The all
option of nyc
includes all source code but does not indicate untouched source code (like BarPage.vue
) as uncovered.
Troubleshooting steps
- [x] still occurring when I put
cache: false
in my nyc config
Environment Information
$ npx envinfo@latest --preset nyc
npx: installed 1 in 0.773s
System:
OS: Linux 4.15 Ubuntu 18.04.5 LTS (Bionic Beaver)
CPU: (8) x64 Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
Memory: 15.50 GB / 31.27 GB
Binaries:
Node: 14.16.1 - /usr/local/bin/node
Yarn: 1.22.5 - /usr/bin/yarn
npm: 6.14.12 - /usr/local/bin/npm
npmPackages:
@babel/register: 7.14.5 => 7.14.5
@istanbuljs/nyc-config-babel: 3.0.0 => 3.0.0
@vue/cli-plugin-babel: 4.5.13 => 4.5.13
babel-core: 7.0.0-bridge.0 => 7.0.0-bridge.0
babel-loader: 8.2.2 => 8.2.2
babel-plugin-istanbul: 6.0.0 => 6.0.0
babel-preset-typescript-vue: 1.1.1 => 1.1.1
source-map-support: 0.5.19 => 0.5.19
typescript: 4.2.2 => 4.2.2
Possibly related issues
- #1372
- https://github.com/istanbuljs/babel-plugin-istanbul/issues/197
- https://github.com/istanbuljs/babel-plugin-istanbul/issues/105
see also https://stackoverflow.com/questions/68194900/cypress-code-coverage-with-nyc-all-option-does-not-report-source-code-as-unco
The problem seems to be caused by https://github.com/cypress-io/code-coverage/issues/207 and its related PR https://github.com/cypress-io/code-coverage/pull/208 .
See Readme of my updated working example for a (not so nice) workaround to add all source files as additional entries in Cypress' webpack preprocessor: https://github.com/toaditoad/code-coverage-mwe
I'm glad to see I'm not the only one confused and dismayed by this behavior. Terrible code coverage comes out looking not-so-bad because of this issue. I would love to see what our actual code coverage is, but I can't get the workaround working - possibly because we're on cypress 7?
any new on this? I'm having the same issue trying to get coverage report for tests using cypress
, I pre-instrument my code with babel-plugin-istanbul
, when trying to generate the report with nyc
even if I use the option all
set to true
, the coverage only shows files that were used when running the cypress
tests any file not used on the tests execution does not appear on the coverage reports.
@toaditoad I also encountered this issue and I believe this is a serious issue of coverage statistics, this causes the code coverage is incorrect. Do you have any workaround to solve this issue for Cypress 12? Thanks in advance.
I'm not even sure if we use nyc any more. Having a quick look at our project, I can tell that vue 3.4.22, cypress-vite 1.5.0, vite 5.2.9 and vite-plugin-istanbul 5.0.0, vitest 1.5.0 is used. Unfortunately, I don't have the time to generate a new minimal working example. However, we have never found a workaround for the originally reported issue. Code coverage seems fine now but so much has changed that I cannot pinpoint the decisive factor.