Global and single file coverage thresholds

[darekkay]

Do you want to request a feature or report a bug?

Feature

What is the current behavior?

Currently, when using both global and path/glob thresholds, the global config combines the coverage for remaining files only (as documented).

What is the expected behavior?

I'll describe my use case. A global threshold is configured as following:

coverageThreshold: {
  global: {
    statements: 85,
    branches: 80,
    functions: 85,
    lines: 85
  }
}

This makes it possible to commit completely untested files as long as the overall coverage is fulfilled. To solve this issue I would like to enforce every file being tested for at least 50%:

coverageThreshold: {
  global: {
    statements: 85,
    branches: 80,
    functions: 85,
    lines: 85
  },
  
  "app/**/*.js": {
    statements: 50,
    branches: 50,
    functions: 50,
    lines: 50
  }
}

However, this won't work as expected. First, running this config will fail if app/ and global point to the same files (which is expected as "If the file specified by path is not found, error is returned."). Second, now the global threshold is overruled by the "local" threshold.

It would be great to have an option to use a global threshold which always applies to all files in addition to glob/path thresholds for single file thresholds.

Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.

• Windows 7
• jest v22.1.4
• node v8.9.1
• npm v5.6.0
• example repository

[thymikee]

@darekkay do you have a repro of that?

[thymikee]

ping @darekkay

[darekkay]

EDIT: Please see my newer comment instead.

---

Sorry for my late response @thymikee , here it is: jest-coverage.

I noticed some differences between jest versions:

In [email protected] it works as expected:

$ jest --coverage
...
------------|----------|----------|----------|----------|----------------|
File        |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------|----------|----------|----------|----------|----------------|
All files   |       80 |      100 |       80 |       80 |                |
 product.js |        0 |      100 |        0 |        0 |            4,7 |
 sum-01.js  |      100 |      100 |      100 |      100 |                |
 sum-02.js  |      100 |      100 |      100 |      100 |                |
 sum-03.js  |      100 |      100 |      100 |      100 |                |
 sum-04.js  |      100 |      100 |      100 |      100 |                |
------------|----------|----------|----------|----------|----------------|
Jest: Coverage for statements (0%) does not meet C:\jest-coverage\app\product.js threshold (50%)
Jest: Coverage for lines (0%) does not meet C:\jest-coverage\app\product.js threshold (50%)
Jest: Coverage for functions (0%) does not meet C:\jest-coverage\app\product.js threshold (50%)

In [email protected] we get this instead:

$ jest --coverage
...
------------|----------|----------|----------|----------|-------------------|
File        |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
------------|----------|----------|----------|----------|-------------------|
All files   |       80 |      100 |       80 |       80 |                   |
 product.js |        0 |      100 |        0 |        0 |               2,5 |
 sum-01.js  |      100 |      100 |      100 |      100 |                   |
 sum-02.js  |      100 |      100 |      100 |      100 |                   |
 sum-03.js  |      100 |      100 |      100 |      100 |                   |
 sum-04.js  |      100 |      100 |      100 |      100 |                   |
------------|----------|----------|----------|----------|-------------------|
Jest: Coverage data for global was not found.
Jest: "C:\jest-coverage\app\product.js" coverage threshold for statements (50%) not met: 0%
Jest: "C:\jest-coverage\app\product.js" coverage threshold for lines (50%) not met: 0%
Jest: "C:\jest-coverage\app\product.js" coverage threshold for functions (50%) not met: 0%

We get Coverage data for global was not found (which, again, is expected as "If the file specified by path is not found, error is returned."). To demontrate the issue a little better, we can delete product.js to get a 100% coverage, which will still cause npm run test to fail:

$ jest --coverage
...
-----------|----------|----------|----------|----------|-------------------|
File       |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-----------|----------|----------|----------|----------|-------------------|
All files  |      100 |      100 |      100 |      100 |                   |
 sum-01.js |      100 |      100 |      100 |      100 |                   |
 sum-02.js |      100 |      100 |      100 |      100 |                   |
 sum-03.js |      100 |      100 |      100 |      100 |                   |
 sum-04.js |      100 |      100 |      100 |      100 |                   |
-----------|----------|----------|----------|----------|-------------------|
Jest: Coverage data for global was not found.

[oorestisime]

I was doing some coverage testing in the gatsby repo and found that calculated global coverage differs when there are files or other folders included included.

i.e

coverageThreshold: {
    global: {
      lines: 42,
      statements: 43,
      functions: 40,
      branches: 42,
    },
  },

This would pass the test, but

coverageThreshold: {
    global: {
      lines: 42,
      statements: 43,
      functions: 40,
      branches: 42,
    },
     "packages/babel-plugin-remove-graphql-queries/src/": { ...coreThresholds },
     "packages/gatsby/src/": { ...coreThresholds },
     "packages/gatsby-plugin-page-creator/src/": { ...coreThresholds },
     "packages/gatsby-image/src/": { ...coreThresholds },
     "packages/gatsby-link/src/": { ...coreThresholds },
     "packages/gatsby-plugin-sharp/src/": { ...coreThresholds },
  },

fails the build because global thresholds dropped to 38.

[boris-meerovich-sp]

Any updates?

[rickhanlonii]

I think the next step is to for someone interested to submit a PR

[gugol2]

I would also appreciate the global coverage not to be contaminated by the more specific file or folder coverages...

[averath]

Why don't you just add:

coverageThreshold: {
    "**/*": {
      "branches": 90,
      "functions": 90,
      "lines": 90,
      "statements": 90
    }
  }

[darekkay]

@averath Because what we need is to have both checks in place, e.g.:

a) The total coverage should be at least 90% b) The per-file coverage should be at least 50%

In other words, it's okay if a single file does not meet the 90% coverage (as long as the overall coverage is at least 90%), but it's not okay if a single file has no coverage at all.

[darshanshah]

I was doing some coverage testing in the gatsby repo and found that calculated global coverage differs when there are files or other folders included included.

i.e

coverageThreshold: {
    global: {
      lines: 42,
      statements: 43,
      functions: 40,
      branches: 42,
    },
  },

This would pass the test, but

coverageThreshold: {
    global: {
      lines: 42,
      statements: 43,
      functions: 40,
      branches: 42,
    },
     "packages/babel-plugin-remove-graphql-queries/src/": { ...coreThresholds },
     "packages/gatsby/src/": { ...coreThresholds },
     "packages/gatsby-plugin-page-creator/src/": { ...coreThresholds },
     "packages/gatsby-image/src/": { ...coreThresholds },
     "packages/gatsby-link/src/": { ...coreThresholds },
     "packages/gatsby-plugin-sharp/src/": { ...coreThresholds },
  },

fails the build because global thresholds dropped to 38.

does that worked for you after that? I am also tring something similar but it doesn't work for me.

[Yilong94]

I have a similar requirement and I did a workaround by chaining commands together and using the coverageThreshold CLI option (which isn't formally documented)

In my package.json:

scripts: {
   ...
   "test": "jest"
   "test:coverage": "npm run test -- --coverage --coverageThreshold='{\"app/**/*.js\":{\"statements\":\"50\"}}' && npm run test -- --coverage --coverageThreshold='{\"global\":{\"statements\":\"80\"}}'"
}
   ...

I will run npm run test:coverage. The first chained command will ensure that individual files committed must have at least 50% code coverage in statements while the second chained command will check that all files in the project must have at least 80% code coverage in statements

[antoniosZ]

could this be related to Windows file paths?

remember, file paths in windows look different than in linux/mac and the OP has windows 7

[darekkay]

@antoniosZ I think the Coverage data for global was not found was a different issue. I could not reproduce it with my example anymore.

However, the main issue is still valid and reproducible both on Windows and a Mac. With the following configuration:

  coverageThreshold: {
    global: {
      statements: 80,
      branches: 80,
      functions: 80,
      lines: 80
    },

    "app/**/*.js": {
      statements: 50,
      branches: 50,
      functions: 50,
      lines: 50
    }
  },

... this example should fail:

------------|---------|----------|---------|---------|-------------------
File        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
------------|---------|----------|---------|---------|-------------------
All files   |   91.67 |       50 |     100 |   91.67 |
 product.js |      75 |       50 |     100 |      75 | 6
 sum-01.js  |     100 |      100 |     100 |     100 |
 sum-02.js  |     100 |      100 |     100 |     100 |
 sum-03.js  |     100 |      100 |     100 |     100 |
 sum-04.js  |     100 |      100 |     100 |     100 |
------------|---------|----------|---------|---------|-------------------

Test Suites: 5 passed, 5 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        1.109 s
Ran all test suites.

Jest reports a successful result, even though the overall branch coverage does not fulfill the global threshold (80%). As discussed, the reason is that Jest uses global for "all files that are unmatched by other threshold patterns" (in this example, there are no other files). In other words, global rather means everythingElse or unmatched.

[github-actions[bot]]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[darekkay]

This is still valid.

[AlbertoSadoc]

I also found the same issue, and probably it's because of the name for global, that looks as if it's for all files but it should actually be named to rest or remaining or unspecified or anything else than global since it's not the threshold for all files

Why don't you just add:

coverageThreshold: {
    "**/*": {
      "branches": 90,
      "functions": 90,

The problem for that is that it will take 90% as the threshold for each file and not all files combined. The only solution I found so far it's to specify all project folders, so the folders can have a combined threshold

[github-actions[bot]]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[darekkay]

This issue is still valid, please un-stale it.