cypress-and-jest icon indicating copy to clipboard operation
cypress-and-jest copied to clipboard

Add a caveat to the documentation on code coverage merging

Open cellog opened this issue 4 years ago • 1 comments

As discussed in https://gitter.im/cypress-io/cypress?at=5f9859d200a0f3369d366d17 the code coverage merging of jest and cypress does not work without some substantial code-smithing.

I have written a new algorithm for merging the coverages, but I did it as an employee at my company, so I need to get permission to open source it.

Of course, there may be other ways to solve this that I don't know of, but the most important thing to note is that istanbul does not instrument files the same way in Cypress + webpack as it does in ts-jest, and code coverage cannot be accurately combined with nyc merge or nyc report (which does merge internally), and that nyc will NOT report errors, it will just report inaccurate coverage.

Hopefully this addition to the README will help prevent other folks from tearing their hair out. Once I understood the problem, it took me half a day to write and fully test a new implementation for merge that properly merges these subset situations and throws an error if there really is an incompatibility, but it does require a pretty deep dive into the istanbul lib to understand what to code.

cellog avatar Oct 31 '20 20:10 cellog

hold off on this. I just discovered the ACTUAL underlying problem: you can't merge code coverage generated using ts-jest with code coverage generated using babel. Using babel-jest results in 1:1 matching of coverage maps. I'll update this PR when I get a chance later this week

cellog avatar Nov 02 '20 20:11 cellog