StopRun(true) is not actually cancelling a hanging test

[CharliePoole]

For a long time, I've been thinking that the framework successfully cancels tests but doesn't report them as cancelled. That's what issue #166 was all about. I don't know if the behavior I see now is newer than that old issue or if it was present back at the time. However, it seems like it has to be fixed first in any case.

The TestCentric GUI uses a test assembly with deliberate infinite loops to test cancellation. Here's the code of one test.

[Test]
public void HangingTest()
{
    TestContext.Progress.WriteLine("Test starting");
    while (true)
    {
        Sleep(3000);
        TestContext.Progress.WriteLine("Test continuing");
    }
}

I've been using this test for a long time and trusting the results, but without the second WriteLine in it. Today, I got suspicious and added that line. What happens when I run my test is that the cancel is issued and returns successfully. Because of issue #166, I don't get a notice so I just assume the cancel worked. But with that new line added I see that the test is never actually cancelled.

I'd like to see someone else confirm this by use of my code or similar code. Of course, you have to call StopRun(true) if you expect the cancel to work in this situation.

[manfred-brands]

@CharliePoole does this work when using the .net framework version? The runner.stop relies on Thread.Abort which isn't available in later versions of .net A similar issue is with the Timeout attribute (#4021) where tests don't actually stop, but just report a timeout. There is no way we can 'kill' a test unless we run the test in a separate process. .net relies on Cooperative multi-tasking. I had suggested passing a CancellationToken to a test. But that won't stop.misbehaving tests.