faker icon indicating copy to clipboard operation
faker copied to clipboard

[BUG] Different coverage percents every time the test run

Open 4k1k0 opened this issue 2 years ago • 1 comments

Describe the bug If you run the test with the coverprofile flag it return different results every time.

To Reproduce

$ for i in {1..10}; do go test -coverprofile=coverage.out .; done

E.g.

$ for i in {1..10}; do go test -coverprofile=coverage.out .; done
ok  	faker	6.565s	coverage: 95.6% of statements
ok  	faker	5.586s	coverage: 94.9% of statements
ok  	faker	6.202s	coverage: 95.6% of statements
ok  	faker	5.363s	coverage: 96.2% of statements
ok  	faker	5.253s	coverage: 96.4% of statements
ok  	faker	5.439s	coverage: 95.9% of statements
ok  	faker	5.360s	coverage: 96.3% of statements
ok  	faker	5.089s	coverage: 96.2% of statements
ok  	faker	6.661s	coverage: 95.1% of statements
ok  	faker	9.051s	coverage: 96.2% of statements

Expected behavior It should return the same value every time.

Desktop (please complete the following information):

  • OS: MacOS Monterrey 12.4
  • go version output: go version go1.11 darwin/amd64

4k1k0 avatar Jul 13 '22 00:07 4k1k0

This is mainly because of the fact that a ton of logic here actually relies on randoming numbers which means that there are different paths that are tested in the coverage each time.

xlanor avatar Jul 18 '22 22:07 xlanor

I set the minimum coverage to 95%. Although all methods are exercised in tests, reaching 100% of coverage will not be possible until we have a major rewrite allowing mocking of many methods using interfaces. For now, I believe this is sufficient, but I'm more than happy to review this in the future if we receive more concerns.

jaswdr avatar Aug 21 '23 18:08 jaswdr