nunit3-vs-adapter icon indicating copy to clipboard operation
nunit3-vs-adapter copied to clipboard
verified publisher icon nunit

VS Test Explorer NUnit error: "TestPlatformFormatException: Filter string ... includes unrecognized escape sequence"

Open Evangelink opened this issue 1 year ago • 8 comments

Describe the bug

When running a parametrized NUnit test where the parameter is the " string, the following error appears in the Test Explorer output (dotnet test works fine):

[17/02/2025 4:25:21.638 pm] [Error] Process: TestProject3\TestProject3\bin\Debug\net8.0\TestProject3.dll, Exception: StreamJsonRpc.RemoteInvocationException: Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.TestPlatformFormatException: Filter string 'TestProject3.Test1.TestMethod1\("\\\"\)' includes unrecognized escape sequence.
   at Microsoft.Testing.Extensions.VSTestBridge.ObjectModel.ContextAdapterBase.GetTestCaseFilter(IEnumerable`1 supportedProperties, Func`2 propertyProvider) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/ObjectModel/ContextAdapterBase.cs:line 43
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_TfsTestCaseFilterExpression() in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 89
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_IsEmpty() in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 91
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.InitializeForExecution(IRunContext runContext, IFrameworkHandle frameworkHandle) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 282
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 120
   at NUnit.VisualStudio.TestAdapter.TestingPlatformAdapter.NUnitBridgedTestFramework.SynchronizedRunTestsAsync(VSTestRunTestExecutionRequest request, IMessageBus messageBus, CancellationToken cancellationToken) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\TestingPlatformAdapter\NUnitBridgedTestFramework.cs:line 40
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.<>c__DisplayClass22_0.<<ExecuteRequestAsync>b__0>d.MoveNext() in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 124
--- End of stack trace from previous location ---
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.ExecuteRequestWithRequestCountGuardAsync(Func`1 asyncFunc) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 145
   at Microsoft.Testing.Extensions.VSTestBridge.VSTestBridgedTestFrameworkBase.ExecuteRequestAsync(ExecuteRequestContext context) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/VSTestBridgedTestFrameworkBase.cs:line 77
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.ExecuteRequestAsync(ITestFramework testFramework, TestExecutionRequest request, IMessageBus messageBus, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 75
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.ExecuteAsync(ITestFramework testFramework, ClientInfo client, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 64
   at Microsoft.Testing.Platform.Hosts.CommonTestHost.ExecuteRequestAsync(ProxyOutputDevice outputDevice, ITestSessionContext testSessionInfo, ServiceProvider serviceProvider, BaseMessageBus baseMessageBus, ITestFramework testFramework, ClientInfo client) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 136
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.ExecuteRequestAsync(RequestArgsBase args, String method, ServiceProvider perRequestServiceProvider) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 507
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.ExecuteRequestAsync(RequestArgsBase args, String method, ServiceProvider perRequestServiceProvider) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 539
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.HandleRequestCoreAsync(RequestMessage message, RpcInvocationState rpcInvocationState) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 433
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.HandleRequestAsync(RequestMessage request, CancellationToken serverClosing) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 339
   at StreamJsonRpc.JsonRpc.InvokeCoreAsync[TResult](RequestId id, String targetName, IReadOnlyList`1 arguments, IReadOnlyList`1 positionalArgumentDeclaredTypes, IReadOnlyDictionary`2 namedArgumentDeclaredTypes, CancellationToken cancellationToken, Boolean isParameterObject)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.RpcClient.RunTestsWithFilterAsync(Guid runId, TestNode[] testNodes, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Threading.ThreadingTools.WithCancellationSlow[T](Task`1 task, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.Clients.TestingPlatformClient.ExecuteRpcCallAsync[T](Func`1 rpcFunction, CancellationToken cancellationToken)
RPC server exception:
:

Steps To Reproduce

See attached file: repro.zip

Expected behavior

No error

Actual behavior

Above error

Reported by @aelij on https://github.com/microsoft/testfx/issues/5047

Evangelink avatar Feb 20 '25 16:02 Evangelink

I recently switched NUnit to MTP per https://docs.nunit.org/articles/vs-test-adapter/NUnit-And-Microsoft-Test-Platform.html and started receiving similar errors in VS2022 17.13.4 w/ NUnit3TestAdapter v5.

Here is a slightly modified version of the provided repro.zip that illustrates the problem I saw: [TestCase(@"FOO\BAR")] public void TestMethod1(string s){ }

The same test runs fine in VS.NET w/o MTP. If I change the test to be non-parameterized, the error goes away w/ MTP.

joschmo80 avatar Mar 21 '25 16:03 joschmo80

Cannot reproduce the issue with the current main (3d0f824243aaaeb85621d3c7dddc92e7a7c45097). Seems like this was (inadvertently?) fixed.

danstur avatar Apr 03 '25 14:04 danstur

Strange. I can't see a possible relevant change between 5.0.0 and master

https://github.com/nunit/nunit3-vs-adapter/compare/V5.0.0...master

Youssef1313 avatar Apr 07 '25 05:04 Youssef1313

Interesting - with the referenced commit it works for the repro.zip project, but I still get the error in our larger solution. Seems a bit non-deterministic or maybe my setup has problems.

Trying to see if I can debug the issue in our main solution.

danstur avatar Apr 07 '25 08:04 danstur

We're getting the same error in https://github.com/gitextensions/gitextensions/pull/12301.

RussKie avatar Apr 07 '25 08:04 RussKie

The issue seems to be related to how the tests are run. One variant works fine, the other crashes. Interestingly enough the same project can one time be run in one way and then the following run the other.

Broken:


[07.04.2025 11:48:48.395 AM]  Operation Dequeue : 'RunSelectedOperation'
[07.04.2025 11:48:48.397 AM]  Building Test Projects
[07.04.2025 11:48:48.413 AM]  Start building 1 containers.
[07.04.2025 11:48:48.413 AM]  Triggering build for 1 IProjectBasedTestContainers.
[07.04.2025 11:48:48.482 AM]  No IBuildableTestContainers were found.
[07.04.2025 11:48:48.482 AM]  Completed building containers.
[07.04.2025 11:48:48.482 AM]  Start updating 1 containers.
[07.04.2025 11:48:48.482 AM]  Updating container E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe.
[07.04.2025 11:48:48.483 AM]  Completed updating containers.
[07.04.2025 11:48:48.483 AM]  DiscoveryOperation<RunSelectedOperation> Changed container count is 0
[07.04.2025 11:48:48.487 AM]  Test run animation has started.
[07.04.2025 11:48:48.488 AM]  Test run progress has started.
[07.04.2025 11:48:48.507 AM]  ========== Starting test run ==========
[07.04.2025 11:48:48.509 AM]  Tests run settings for E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe:
 <RunSettings>
  <BoostTestInternalSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <VSProcessId>33260</VSProcessId>
  </BoostTestInternalSettings>
  <RunConfiguration>
    <ResultsDirectory>E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\TestResults</ResultsDirectory>
    <SolutionDirectory>E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\</SolutionDirectory>
    <CollectSourceInformation>False</CollectSourceInformation>
  </RunConfiguration>
  <GoogleTestAdapterSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <SolutionSettings>
      <Settings />
    </SolutionSettings>
    <ProjectSettings />
  </GoogleTestAdapterSettings>
</RunSettings>.
[07.04.2025 11:48:48.510 AM]  Launching test runner 'E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe'
[07.04.2025 11:48:48.511 AM]  Logging diagnostics under 'E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\Log'
[07.04.2025 11:48:48.512 AM]  Created TCP listener with port 0 for program 1
[07.04.2025 11:48:48.513 AM]  Starting test runner 'E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe  --server --diagnostic --diagnostic-verbosity Debug --diagnostic-output-directory "E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\Log" --results-directory "E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\TestResults" --client-port 56847'
[07.04.2025 11:48:49.179 AM]  Received new TCP connection, process: 43492
[07.04.2025 11:48:49.180 AM]  Capabilities: SupportsDiscovery: True, SupportsTestExecution: False, SupportsMultiRequest: False, SupportsVsTestProvider: True, SupportsAttachments, TrueMultiConnectionProvider: False
[07.04.2025 11:48:49.183 AM]  Connected to main test runner node
[07.04.2025 11:48:49.183 AM]  Connected to test runner 'E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe'
[07.04.2025 11:48:49.198 AM]  Connecting to client host '127.0.0.1' port '56847'
[07.04.2025 11:48:49.312 AM]  NUnit Adapter 5.1.0.0: Test execution started
[07.04.2025 11:48:49.342 AM] [Error] Process: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe, Exception: StreamJsonRpc.RemoteInvocationException: Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.TestPlatformFormatException: Filter string 'ProjectGraphAnalysis.Tests.Tia.ModifiedFilesProviderTests.Given_git_encoded_file_then_normal_path_is_returned\("Foo/A/B/Application Note - OurProject\\\302\\\256 AutoAnalysis_OurProject\\\302\\\256 LE, OurProject\\\302\\\256 neXT - Service.pdf","Foo/A/B/Application Note - OurProject® AutoAnalysis_OurProject® LE, OurProject® neXT - Service.pdf"\)' includes unrecognized escape sequence.
   at Microsoft.Testing.Extensions.VSTestBridge.ObjectModel.ContextAdapterBase.GetTestCaseFilter(IEnumerable`1 supportedProperties, Func`2 propertyProvider) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/ObjectModel/ContextAdapterBase.cs:line 53
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_TfsTestCaseFilterExpression() in E:\source\oss\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 89
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_IsEmpty() in E:\source\oss\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 91
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.InitializeForExecution(IRunContext runContext, IFrameworkHandle frameworkHandle) in E:\source\oss\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 279
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in E:\source\oss\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 120
   at NUnit.VisualStudio.TestAdapter.TestingPlatformAdapter.NUnitBridgedTestFramework.SynchronizedRunTestsAsync(VSTestRunTestExecutionRequest request, IMessageBus messageBus, CancellationToken cancellationToken) in E:\source\oss\nunit3-vs-adapter\src\NUnitTestAdapter\TestingPlatformAdapter\NUnitBridgedTestFramework.cs:line 38
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.<>c__DisplayClass22_0.<<ExecuteRequestAsync>b__0>d.MoveNext() in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 166
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.<ExecuteRequestWithRequestCountGuardAsync>d__24.MoveNext() in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 188
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Extensions.VSTestBridge.VSTestBridgedTestFrameworkBase.<ExecuteRequestAsync>d__19.MoveNext() in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/VSTestBridgedTestFrameworkBase.cs:line 91
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.<ExecuteRequestAsync>d__16.MoveNext() in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.<ExecuteAsync>d__15.MoveNext() in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 63
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Platform.Hosts.CommonTestHost.<ExecuteRequestAsync>d__12.MoveNext() in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 137
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.<ExecuteRequestAsync>d__43.MoveNext() in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 511
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.<ExecuteRequestAsync>d__43.MoveNext() in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 534
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.<HandleRequestCoreAsync>d__42.MoveNext() in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 428
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.<HandleRequestAsync>d__40.MoveNext() in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 334
   at StreamJsonRpc.JsonRpc.InvokeCoreAsync[TResult](RequestId id, String targetName, IReadOnlyList`1 arguments, IReadOnlyList`1 positionalArgumentDeclaredTypes, IReadOnlyDictionary`2 namedArgumentDeclaredTypes, CancellationToken cancellationToken, Boolean isParameterObject)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.RpcClient.RunTestsWithFilterAsync(Guid runId, TestNode[] testNodes, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Threading.ThreadingTools.WithCancellationSlow[T](Task`1 task, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.Clients.TestingPlatformClient.ExecuteRpcCallAsync[T](Func`1 rpcFunction, CancellationToken cancellationToken)
RPC server exception:
: 

[07.04.2025 11:48:49.343 AM]  Attachments processing not needed
[07.04.2025 11:48:49.345 AM]  Test run animation has stopped.
[07.04.2025 11:48:49.345 AM]  Test run progress has stopped.
[07.04.2025 11:48:49.345 AM]  ========== Test run aborted: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 832,9 ms ==========

Working:

[07.04.2025 11:45:53.755 AM]  Operation Dequeue : 'RunSelectedOperation'
[07.04.2025 11:45:53.776 AM]  Building Test Projects
[07.04.2025 11:45:53.777 AM]  Start building 1 containers.
[07.04.2025 11:45:53.782 AM]  Loading Project e38da69b-a51d-4eb2-2fc9-34262515115a [2] => 
===
Id: e38da69b-a51d-4eb2-2fc9-34262515115a
ProjectFilePath: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\ProjectGraphAnalysis.Tests.csproj
DefaultOutputPath: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe
DefaultTargetFramework: net48
ProjectName: ProjectGraphAnalysis.Tests
DefaultCapabilities: Managed,CPS,TestContainer,CSharp,TestingPlatformServer
IsAppContainer: False
IsCpsProject: True
IsMiscellaneous: False
HasTestContainerCapability: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe: True
HasTestingPlatformServerCapability: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe: True
IsCSharpOrVBProject: True

[07.04.2025 11:45:53.782 AM]  Triggering build for 1 IProjectBasedTestContainers.
[07.04.2025 11:45:53.852 AM]  No IBuildableTestContainers were found.
[07.04.2025 11:45:53.852 AM]  Completed building containers.
[07.04.2025 11:45:53.852 AM]  Start updating 1 containers.
[07.04.2025 11:45:53.852 AM]  Updating container E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe.
[07.04.2025 11:45:53.852 AM]  Completed updating containers.
[07.04.2025 11:45:53.852 AM]  DiscoveryOperation<RunSelectedOperation> Changed container count is 0
[07.04.2025 11:45:53.853 AM]  Test run animation has started.
[07.04.2025 11:45:53.853 AM]  Test run progress has started.
[07.04.2025 11:45:53.862 AM]  Loading Project 680769fe-8b8f-d05b-c58f-17883c1045b1 [2] => 
===
Id: 680769fe-8b8f-d05b-c58f-17883c1045b1
ProjectFilePath: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis\ProjectGraphAnalysis.csproj
DefaultOutputPath: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\runtime\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.exe
DefaultTargetFramework: net48
ProjectName: ProjectGraphAnalysis
DefaultCapabilities: Managed,CPS,CSharp
IsAppContainer: False
IsCpsProject: True
IsMiscellaneous: False
HasTestContainerCapability: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\runtime\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.exe: False
HasTestingPlatformServerCapability: E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\runtime\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.exe: False
IsCSharpOrVBProject: True

[07.04.2025 11:45:53.872 AM]  ========== Starting test run ==========
[07.04.2025 11:45:53.874 AM]  Tests run settings for E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe:
 <RunSettings>
  <BoostTestInternalSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <VSProcessId>33260</VSProcessId>
  </BoostTestInternalSettings>
  <RunConfiguration>
    <ResultsDirectory>E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\TestResults</ResultsDirectory>
    <SolutionDirectory>E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\</SolutionDirectory>
    <CollectSourceInformation>False</CollectSourceInformation>
  </RunConfiguration>
  <GoogleTestAdapterSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <SolutionSettings>
      <Settings />
    </SolutionSettings>
    <ProjectSettings />
  </GoogleTestAdapterSettings>
</RunSettings>.
[07.04.2025 11:45:54.388 AM]  NUnit Adapter 5.1.0.0: Test execution started
[07.04.2025 11:45:54.650 AM]  Running selected tests in E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe
[07.04.2025 11:45:54.857 AM]     NUnit3TestExecutor discovered 20 of 20 NUnit test cases using Current Discovery mode, Non-Explicit run
[07.04.2025 11:45:56.421 AM]  [11:45:55.1926] [Info] [ProjectParser] Using Visual Studio Professional 2022 with MSBuildPath C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Current\Bin.

The difference seems to be: [07.04.2025 11:45:54.650 AM] Running selected tests in E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe vs. [07.04.2025 11:48:48.513 AM] Starting test runner 'E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\ProjectGraphAnalysis.Tests.exe --server --diagnostic --diagnostic-verbosity Debug --diagnostic-output-directory "E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\ProjectGraphAnalysis.Tests\bin\Debug\net48\Log" --results-directory "E:\source\OurProject\src\BuildTools\ProjectGraphAnalysis\TestResults" --client-port 56847'

So in some situations VS will execute the exact source code differently, which can mask the bug.

danstur avatar Apr 07 '25 09:04 danstur

It's all about whether you are executing all tests or selecting specific test(s) from Test Explorer.

Youssef1313 avatar Apr 07 '25 11:04 Youssef1313

It's all about whether you are executing all tests or selecting specific test(s) from Test Explorer.

I'd have thought so too, but I'm clicking on the top level of the project in Test Explorer. Also occurs when clicking "Run tests" on the solution node in the UI. It's not deterministic as far as I can see.

In any case, not a big problem as soon as one knows about it.

Debugging the actual problem is a bit more involved because it seems to be an interaction between TestFx and nunit and at the time the error surfaces the actual parsing has long occurred and it's just storing the error in a field.

danstur avatar Apr 07 '25 12:04 danstur

Related: #1332

cbersch avatar Oct 21 '25 07:10 cbersch

This issue is solved with PR #1341 .

You can test it with the package from myget (See links bottom of #1332)

OsirisTerje avatar Oct 25 '25 11:10 OsirisTerje