Potential error in merging code coverage

[bahmutov]

While trying to split some specs in PR

I have noticed that the coverage fell but then went back up without any code changes. Compare

Coverage 93.95% run https://circleci.com/gh/cypress-io/cypress-example-realworld/374#artifacts/containers/0 report https://374-193011024-gh.circle-artifacts.com/0/root/project/coverage/lcov-report/index.html

and then next run with exactly the same specs

Coverage 95.2% build https://377-193011024-gh.circle-artifacts.com/0/root/project/coverage/lcov-report/index.html report https://circleci.com/gh/cypress-io/cypress-example-realworld/377#artifacts/containers/0

Hmm, maybe the merge does not merge files if they are not created? Maybe related to #54

[joaquinricci]

Hi @bahmutov !

Just an idea, because I have a similar workflow as yours for coverage and my problem was the same:

The test coverage was not deterministic when running tests in parallel.

My thoughts are that maybe, and since they are in parallel, a coverage out.json gets overwritten by another parallel result before it gets copied into its unique file name.

I am just starting to dive in this... I guess I will be back here if I find something, but you might have already fixed this. For now I have just solved it by running them sequentially, but this is clearly not a good solution as the tests grow.