visualstudio.xunit icon indicating copy to clipboard operation
visualstudio.xunit copied to clipboard

In vstest mode the xunit test projects keep running, even after testhost is closed

Open drognanar opened this issue 4 months ago • 7 comments

Repro steps

This can be for example tested on https://github.com/microsoft/vs-streamjsonrpc, by forcing vstest mode in the test project:

<DisableTestingPlatformServerCapability>true</DisableTestingPlatformServerCapability>
<TestingPlatformDotnetTestSupport>false</TestingPlatformDotnetTestSupport>
<UseMicrosoftTestingPlatformRunner>false</UseMicrosoftTestingPlatformRunner>

If a user triggers a run in test explorer and then stops it mid-run, the test runner does not finish running it immediately. It keeps running until completion. This causes file locking issues to appear.

Expected

vstest and testhosts pass in the parentprocessid, so that the child process closes immediately as soon as the parent closes. if xunit test process would similarly pass in the parent process id, it could also detect parent closing and close immediately.

drognanar avatar Aug 12 '25 09:08 drognanar

If a user triggers a run in test explorer and then stops it mid-run, the test runner does not finish running it immediately. It keeps running until completion. This causes file locking issues to appear.

We rely on v3 tests to respond to the CancellationToken in TestContext.Current.CancellationToken for timely shutdown. On the command line, Ctrl+C once will request an orderly shutdown; Ctrl+C a second time will immediately terminate. There is no "immediately terminate" version available via VSTest, and we've fielded enough issues with not being able to orderly cleanup that we would not consider switching our default behavior at this point.

bradwilson avatar Aug 12 '25 17:08 bradwilson

vstest and testhosts pass in the parentprocessid

I don't know what this means.

bradwilson avatar Aug 12 '25 17:08 bradwilson

parentprocessid is a command line switch that passes in the parent's process id. for instance if vstest.console.exe has PID 2500, it will start the testhost by passing it --parentprocessid 2500 on the command line. then if the testhost detects that a process with PID 2500 has exited, it will perform an immediate termination.

Similarly MTP has the following parent process listener https://github.com/microsoft/testfx/blob/d64da42485ebcccfa5191c543c91a35f22f070d0/src/Platform/Microsoft.Testing.Platform/Helpers/NonCooperativeParentProcessListener.cs#L32, where if the parent process passes a command line switch, it will automatically stop if the parent terminates. In MTP's case if the parent's terminated through any means the child will shutdown immediately, rather than try to cancel the test run gracefully.

drognanar avatar Aug 12 '25 19:08 drognanar

I'm trying to decide how much I care to solve this problem, since it requires changes to both v3 core and the VSTest adapter, given that VSTest is basically dead technology now.

Why aren't you (Microsoft teams) all using MTP?

bradwilson avatar Aug 13 '25 19:08 bradwilson

We’re been continuously converting as many repos/projects to MTP as we can.

However there are still some feature gaps like data collectors/coverlet not working where we’re not yet able to do the conversion to MTP. As a result not all solutions can be converted yet.

drognanar avatar Aug 15 '25 16:08 drognanar

With latest xunit mtp-v2 in latest vs2026, I also have the bug of tests constantly running even if they are done and successful..

alexTr3 avatar Nov 25 '25 22:11 alexTr3

I'm wondering if this issue is related to this: https://github.com/xunit/xunit/issues/3447

bradwilson avatar Nov 25 '25 23:11 bradwilson