[Bug]: Coverage report invalid coverage for files when test file changes

[sandinosaso]

Version

29.0.1

Steps to reproduce

1. Clone my repo at: https://github.com/sandinosaso/jest-coverage-bug-reproduction.git
2. Run yarn install
3. Change something (add an space at the end) to the file src/b.test.js
4. Run yarn test --coverage --changedSince=origin/master --shard=1/1

Expected behavior

I expect either:

1. Because the require of a.js in b.test.js determined that coverage should be show for that a.js file I would expect all the tests of a.js are run (i.e the file a.test.js is run because has tests that belong to a.js and that we are going to coverage report
2. Do not show any coverage report for a.js file because this file was not changed

Actual behavior

In the console of the left the coverage report shows 100% for the only changed file (b.js) - This is the expected behavior

In the console of the right side you see running the same command after adding the require('./a.js') see that now the coverage is also shown for the a.js file and it is not 100% percent as expected:

Additional context

Somehow may be related, there is a difference when we run yarn test FILEPATH in that case we see also that (not matter if there is a require o not in the test file testing) then the coverage it not shown complete for the file as you can see above:

Environment

System:
    OS: macOS 10.15.7
    CPU: (8) x64 Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz
  Binaries:
    Node: 14.15.5 - ~/.nvm/versions/node/v14.15.5/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v14.15.5/bin/yarn
    npm: 6.14.11 - ~/.nvm/versions/node/v14.15.5/bin/npm
  npmPackages:
    jest: ^29.0.1 => 29.0.1

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[cdriscol]

My team is also effected by this - @sandinosaso, did you come up with any workarounds? We're having a tough time enforcing coverage thresholds because when test files change we end up with a lot of files (imported from tests) not getting covered because their corresponding tests are not run.

[snehalDH]

@cdriscol We handled this scenario by marking imported files as changed. To do this we created a method which merges imported files and changed files. This set of files is then passed to jest method this.findRelatedTests(mergedFiles, collectCoverage); which collects coverage from imported files as well.

async findTestRelatedToChangedAndImportedFiles(changedFilesInfo, collectCoverage) {
    if (!hasSCM(changedFilesInfo)) {
      return {
        noSCM: true,
        tests: []
      };
    }

    const {changedFiles} = changedFilesInfo;

    const importedFiles = await this.findRelatedSourcesFromTestsInChangedFiles(changedFilesInfo); //imported files which are marked for coverage
    const mergedFiles = new Set([...changedFiles, ...importedFiles]);

    return this.findRelatedTests(mergedFiles, collectCoverage);
  }

call this method in -

async _getTestPaths(globalConfig, changedFiles) {
    if (globalConfig.onlyChanged) {
      if (!changedFiles) {
        throw new Error('Changed files must be set when running with -o.');
      }
      return this.findTestRelatedToChangedAndImportedFiles(
        changedFiles,
        globalConfig.collectCoverage
      );
    }
    ...
    ...
    ...
}

Only drawback is if the code is tightly coupled this will run lot of test cases as it calls resolveInverseModuleMap() internally which jest use to resolve inverse dependency (inverse transitive dependency).

You can use patch-package to apply changes to your repo.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[sjnho]

@snehalDH Hello , can you tell me which file are the _getTestPaths function in ? I 'm not familiar with jest source code. Thanks a lot.

[xabierlameiro]

any solution for this point?

[snehalDH]

@sjnho , you can find it _getTestPaths here .

[vshevchenko12]

Someone received the expected result? I tried to use a solution by @snehalDH But unfortunately doesn't work for me

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[snehalDH]

shevchenko12, can you elaborate what exactly didn't work for you? I can try to help.

[vshevchenko12]

@snehalDH I tried again and it works now. Thank you very much for your solution!

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 30 days with no activity. Please open a new issue if the issue is still relevant, linking to this one.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.