Add unit tests for PerfDiff regression strategies (Part 1: Mean-based strategies)

[coderabbitai[bot]]

trafficstars

Overview

This issue covers testing the mean-based regression detection strategies in PerfDiff.

Scope

Components to Test

• MeanPercentageRegressionStrategy - Dual threshold logic (relative + absolute)
• MeanWallClockRegressionStrategy - Mean execution time analysis
• RegressionStrategyHelper - Shared regression detection logic

Test Scenarios

1. MeanPercentageRegressionStrategy

   • Relative threshold (5% ratio) detection
   • Absolute threshold (0.5ms) detection
   • Both thresholds must be exceeded for regression
   • Handle better/worse performance classifications
   • Exclude infinity ratios correctly
   • Different statistical scenarios (faster, slower, same)

2. MeanWallClockRegressionStrategy

   • 100ms threshold detection
   • Mean execution time comparison logic
   • Time unit conversion (nanoseconds to milliseconds)
   • Statistical significance testing

3. RegressionStrategyHelper

   • Generic threshold-based regression detection
   • Different threshold types (ratio, milliseconds)
   • Metric selector functions
   • Display value selector functions
   • Logging output verification

Test Scenarios by Strategy

• No regression detected - Performance within acceptable thresholds
• Regression detected - Performance degraded beyond thresholds
• Performance improvement - Better performance, no regression
• Edge cases - NaN values, zero statistics, missing data
• Complex scenarios - Mixed results (some better, some worse)

Test Data Requirements

• Create benchmark data sets with known statistical properties
• Design data to trigger specific threshold conditions
• Include edge cases (extreme values, missing statistics)
• Cover all equivalence test conclusions

Acceptance Criteria

• [ ] MeanPercentageRegressionStrategy has 100% test coverage
• [ ] MeanWallClockRegressionStrategy has 100% test coverage
• [ ] RegressionStrategyHelper has 100% test coverage
• [ ] All threshold conditions are tested (met, exceeded, not met)
• [ ] Edge cases and mathematical boundary conditions are covered
• [ ] Logging output is verified for regression detection scenarios
• [ ] Performance improvement scenarios don't trigger false regressions

Implementation Notes

• Mock ILogger to verify log messages
• Use Theory tests for different threshold scenarios
• Create helper methods to generate test benchmark data
• Test mathematical edge cases thoroughly (NaN, Infinity, zero)
• Verify threshold calculations match expected statistical logic

Related

• Parent issue: #[MAIN_ISSUE_NUMBER]
• Components: src/tools/PerfDiff/BDN/Regression/MeanPercentageRegressionStrategy.cs, src/tools/PerfDiff/BDN/Regression/MeanWallClockRegressionStrategy.cs, src/tools/PerfDiff/BDN/Regression/RegressionStrategyHelper.cs