XUnit gives error level message when test fails

[Cosifne]

When tests fail, XUnit will log an error level message to Visual Studio, which is strange, because 'error' is often used to indicate some unexpected exceptions happen. The results of tests should probably be listed as 'Informational' or other logging levels just like the test details.

[bradwilson]

Out of curiosity: why does it matter?

[Cosifne]

Out of curiosity: why does it matter?

Hello Brad, So currently if you run a failed Xunit test in VS, you will see this at the bottom of Visual Studio, It is shown because the XUnit error level message. I don't have a good way to distinguish the XUnit error messages from other error messages (like the error messages from vstest console) happens during the test run, so it would be ideal if XUnit could change this '[5/1/2019 5:23:30 PM Error] [xUnit.net 00:00:00.66] XUnitTestProject2.UnitTest1.Test [FAIL]' to Informational like other test details, otherwise user will see the warning message again and agin

[AbhitejJohn]

@clairernovotny : Any chance we could schedule a fix for this in the near future? There's active work to surface infra errors in the test workflows better and this ends up being a false positive. It would be great if this message was logged under an "informational" mode like Shen mentions above. It seems like xunit Core might be sending this out although the exact location evades my github search results.

[clairernovotny]

I'll accept a PR with a fix but I'm otherwise unlikely to have time to look at it otherwise.

[clairernovotny]

I believe all of the logging flows through here but I haven't debugged it lately: https://github.com/xunit/visualstudio.xunit/blob/277a7a4754ebb37aeffb0e2b8ea64cc27104a3a6/src/xunit.runner.visualstudio/Utility/LoggerHelper.cs#L8-L12

[bradwilson]

I'm unable to reproduce this with VS 2022 17.5.5. All failed tests just show up as fail tests; there are no banners anywhere that I can see.

This is the output I see in the Tests pane of the Output window:

========== Starting test run ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 7.0.5)
[xUnit.net 00:00:00.22]   Starting:    visualstudio-127
[xUnit.net 00:00:00.26]     visualstudio_127.UnitTest1.Test1 [FAIL]
[xUnit.net 00:00:00.26]       Assert.True() Failure
[xUnit.net 00:00:00.26]       Expected: True
[xUnit.net 00:00:00.26]       Actual:   False
[xUnit.net 00:00:00.26]       Stack Trace:
[xUnit.net 00:00:00.26]         C:\Dev\repro\visualstudio-127\UnitTest1.cs(8,0): at visualstudio_127.UnitTest1.Test1()
[xUnit.net 00:00:00.26]            at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[xUnit.net 00:00:00.26]            at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
[xUnit.net 00:00:00.26]   Finished:    visualstudio-127
========== Test run finished: 1 Tests (0 Passed, 1 Failed, 0 Skipped) run in 273 ms ==========

If you're still experiencing this, please provide a full repro project and exact repro steps. Thanks!

[bradwilson]

Assuming with no response that this how now been fixed in 2022. Please open a new issue if this is still a problem. Thanks!