dotnet
Helix test failures are double counted
Describe the bug When a helix test fails gracefully the failure should be reported and the runner should return 0.
To Reproduce Examine a normal test failure for a build. You'll see the test failure reported and the "work item" for the test. The latter is reported as a failure since the runner returned a non-zero exit value.
Expected behavior Only the failing test is reported.
Screenshots, Code, Sample Projects
.
Example build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=583517&view=ms.vss-test-web.build-test-results-tab&runId=14034000&resultId=100083&paneView=debug
Looks like xunit.runner will return either 0 or 1 for normal cases: https://github.com/xunit/xunit/blob/c10b7a2884edcb69a4afe0af6bf241f5565e99e9/src/xunit.console/ConsoleRunner.cs#L101C24-L101C33
I think we just need to swallow the 1 value.