imperative icon indicating copy to clipboard operation
imperative copied to clipboard

Random codecov diffs

Open mheuzey opened this issue 4 years ago • 0 comments

When opening #458, I noticed that the codecov report initially showed some lines losing coverage in a file that I had not touched (CommandResponse.ts). The diff disappeared and reappeared as I made minor adjustments to the PR (no code changes). Looking into this topic of random codecov diffs, I found a thread which suggested that the tests might be non-deterministic. So I ran the unit test for that file on my local machine a few times, and it does look like the coverage changes between runs.

image

I'm not an expert on tests, but I figured I would bring this up so someone with more knowledge could verify whether something in the test suite could be causing random coverage changes. I don't know how widespread this problem is, but looking over the last few PRs, I see that #456 and #435 both have the same coverage diff as mine, and both PRs made small changes that shouldn't have affected coverage.

mheuzey avatar Nov 11 '20 20:11 mheuzey