newrelic
Troubleshoot SimpleCov branch testing
Is your feature request related to a problem? Please describe.
SimpleCov was added to the Ruby agent to provide code coverage analysis. SimpleCov originally reported line coverage when our CI runs, but a feature was added to report on branch coverage as well. However, reported branch coverage results seem too low, and further investigation is needed to understand how we might change SimpleCov in our agent to provide more accurate testing results. The issue likely is related to multiverse testing interacting with unit tests.
Our branch coverage threshold is currently set to 30% so that we aren't continuously failing CI (reported branch coverage is ~35%). Once branch coverage is reporting realistic results, we will raise SimpleCov'sfailedThresholdBranch inside ci.yml to about 5 less than the reported results.
Feature Description
Accurate SimpleCov branch testing.
Priority
Please help us better understand this feature request by choosing a priority from the following options: Really Want
Some additional context: We're concerned that the mechanism that stitches together our multiple test runs and/or multiple CI steps is not accurately reporting the simplecov coverage numbers.
First step: Understand if this hypothesis is correct.
