vstest icon indicating copy to clipboard operation
vstest copied to clipboard

Exception thrown in TestHostTraceListener.Fail does not include stack trace or method name

Open nagya opened this issue 1 year ago • 0 comments

Description

When a Trace.Assert or similar method fails in an xUnit test run with VSTest, the result (in VS or CI logs) only has a nondescript error message, but no stack trace, file/line info, or method name, to aid diagnostics.

I suspect trying to explicitly include a trace in the exception being thrown here, causes this symptom. Manually overriding the trace listener and throwing an exception from its Fail method, without gathering a separate trace first does include a stack trace in the result (example in Expected behavior below.)

Steps to reproduce

  • Create xUnit test
  • Add a Trace.Assert(false) to test code
  • Run test in VS

Expected behavior

Result contains full stack trace with line numbers and method name, e.g.:

========== Starting test run ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.0+6438bb880a (64-bit .NET 8.0.7)
[xUnit.net 00:00:00.04]   Starting:    UnitTests
[xUnit.net 00:00:00.08]     UnitTests.A.B [FAIL]
[xUnit.net 00:00:00.08]       Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException : Method B failed with '', and was translated to Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException to avoid terminating the process hosting the test : 
[xUnit.net 00:00:00.08]       
[xUnit.net 00:00:00.08]       Stack Trace:
[xUnit.net 00:00:00.08]            at System.Diagnostics.TraceInternal.Fail(String message)
[xUnit.net 00:00:00.08]         D:\...\A.cs(485,0): at A.B()
[xUnit.net 00:00:00.08]            at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[xUnit.net 00:00:00.08]            at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
[xUnit.net 00:00:00.08]   Finished:    UnitTests
========== Test run finished: 1 Tests (0 Passed, 1 Failed, 0 Skipped) run in 95 ms ==========

Actual behavior

Result contains only this. Note that not only is the stack trace missing, but the method name was not inferred either.

========== Starting test run ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.0+6438bb880a (64-bit .NET 8.0.7)
[xUnit.net 00:00:00.04]   Starting:    UnitTests
[xUnit.net 00:00:00.44]     UnitTests.A.B [FAIL]
[xUnit.net 00:00:00.44]       Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException : Method <method> failed with '', and was translated to Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException to avoid terminating the process hosting the test.
[xUnit.net 00:00:00.45]   Finished:    UnitTests
========== Test run finished: 1 Tests (0 Passed, 1 Failed, 0 Skipped) run in 457 ms ==========

Environment

  • xunit (2.9.0)
  • xunit.runner.visualstudio (2.8.2)
  • Microsoft.NET.Test.Sdk (17.11.0-release-24352-06)

nagya avatar Jul 23 '24 01:07 nagya