UnhandledExceptionEventHandler in MSTest doesn't show the exception

[vik542]

Description

Using the UnhandledExceptionEventHandler in MSTest doesn't show any exception being caught.

Steps to reproduce

• Create a simple UnitTest project which could be either MSTest (.NET core) or UnitTest Project(.NET Framework)
• Attach an UnhandledException event handler to the AppDomain
• Create any kind of null reference or an InvalidOperation Exception and try to add it to your logger or Console Write the caught exception.

[TestClass]
public class UnitTest1
{
    [AssemblyInitialize]
    public static void AssemblyInitialize(TestContext testContext)
    {            
        AppDomain currentDomain = AppDomain.CurrentDomain;
        currentDomain.UnhandledException += currentDomain_UnhandledException;                     
    }

    static void currentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
    {
        Exception ex = e.ExceptionObject as Exception;
        if (ex != null)
        {
            Console.WriteLine("==========This is from Unhandled Exception EventHandler====================");
            Console.WriteLine("===============================");
            Console.WriteLine("Test failed:" + ex);
        }            
    }
    [TestInitialize]
    public void Initialize()
    {

    }

    [TestCleanup]
    public void Cleanup()
    {

    }
             
    [TestMethod]
    public void RaiseEvents()
    {        
        var list = new List<string>(); //empty list
        var firstItem = list.First().Contains("a");            
    }

    [AssemblyCleanup]
    public static void AssemblyCleanUp()
    {

    }
} 

Expected behavior

Should display the Unhandled exception

Actual behavior

Nothing is displayed in the output window under TestExplorer

Environment

• Window 10 Pro
• Build version of vstest.console: 15.9.1
• Microsoft.NET.Sdk 16.4.0
• MSTest.TestAdapter 2.0.0
• MSTest.TestFramework 2.0.0

[vik542]

Can someone please reply on this.

[singhsarab]

@vikmy What is the intention here ?

[vik542]

@vikmy What is the intention here ?

Below is is just one of the example scenarios. I'm using MSTest for selenium :

In some of my tests I'm expecting that a collection contains some Ui elements(like dropdown values) but due to some functionality or due to some test data, collection will be either empty or it doesn't contain the required data. In such cases when I'm trying to access the elements MSTest Platform throws Invalid Operation exception. Control or flow of execution goes directly to TestCleanup. When I wanted to access/capture the exception for my reports. I do not have any mechanism to capture such exceptions. TestContext has a property which tells me only the outcome of the test but it doesn't give any exception information.

If you have the same scenario in some console projects you can capture the exceptions but the same is not available in MSTest.

[elgatov]

Use case is exactly the same as mine, and a lot of other people using Selenium, for that matter.

[Evangelink]

Hey folks,

Sorry it took so much time to investigate this issue.

MSTest framework, as most test framework, is catching "all" exceptions that occurs (on the same thread) during "test method" execution and consider test as failed. Because all the exceptions are caught, it's expected that the AppDomain.UnhandledException doesn't trigger as there is no uncaught exception. If the exception happens on a different thread then the UnhandledException will be triggered (see example hereafter):

namespace TestProject12 {
    [TestClass]
    public class UnitTest10 {
        [AssemblyInitialize]
        public static void AssemblyInitialize(TestContext testContext) {
            AppDomain currentDomain = AppDomain.CurrentDomain;
            currentDomain.UnhandledException += currentDomain_UnhandledException;
        }
        
        static void currentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e) {
            Exception ex = e.ExceptionObject as Exception;
            if (ex != null) {
                Console.WriteLine("==========This is from Unhandled Exception EventHandler====================");
                Console.WriteLine("===============================");
                Console.WriteLine("Test failed:" + ex);
            }
        }

        [TestInitialize]
        public void Initialize() {

        }

        [TestCleanup]
        public void Cleanup() {
            Thread.Sleep(5000);
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();
        }

        [TestMethod]
        public void RaiseEvents() {
            var list = new List<string>(); //empty list
            new Thread(() => 
            {
                Thread.Sleep(1000); // just to be sure exception happens outside of method execution
                var firstItem = list.First().Contains("a"); 
            }).Start();
        }

        [AssemblyCleanup]
        public static void AssemblyCleanUp() {

        }
    }
}

My understanding of the above scenario is that you simply want to change some manual report or log, in which case, I think your request is similar to "How to get exception from test in TestCleanup?" #293.

If that's not the case, could you please provide more information?

[microsoft-github-policy-service[bot]]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 10 days of this comment.

[elgatov]

If that's not the case, could you please provide more information?

I will give you an example use case for people using Selenium and MSTest for web testing (although there are a lot of other frameworks that execute similar tests flows).

Create a new user and make sure the new user can log in the platform

So you divide the test in 3 steps

1. Log in the platform and create the new user
2. Log in the platform with the new user
3. Delete the user created

You translate this flow into MSTest:

1. [TestInitialize] public void CreateNewUser() { ... }
2. [TestMethod] public void LogInPlatform() { ... }
3. [TestCleanup] public void DeleteUser() { ... }

If the [TestMethod] fails you can check in [TestCleanup] and log it (and take a screenshot of the page, which is usually very helpful). However, if it fails in [TestInitialize] or [TestCleanup] you cannot check it and are forced to wrap the whole method in a try-catch to make sure everything went OK or take a screenshot otherwise.

The whole thing is cumbersome and it would be great if we could take another route. UnhandledException was, in theory, a great way of achieving this: you intercept the exception, log and take screenshot et cetera, and you resume executing.

[microsoft-github-policy-service[bot]]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 10 days of this comment.

[ChristoWolf]

Bump.

[microsoft-github-policy-service[bot]]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 10 days of this comment.

[elgatov]

i must say i am a hugely disappointed by this decision

[Evangelink]

I think you have some misconceptions about the unhanded handlers work. As we are directly catching the exception it would never bubble up in this path and this is the correct behavior. I haven't done tests but would be hugely surprised if this was working any different in other test frameworks.

[elgatov]

As we are directly catching the exception it would never bubble up in this path and this is the correct behavior.

would it be feasible for MSTest to provide an UnhandledException event handler at the TestClass level?

[Evangelink]

Technically we could provide some handler (cannot be Unhandled because the exceptions are handled) but what would you do? What I mean is that eventing system is a trigger not awaited so by the time you act on it, it's possible we have disposed the class, or run some other method that would change the state.

Now if you would like to easily inject some TestCleanup method behavior in all classes, this is something that would be different, would be part of the workflow (and awaited) giving you a change to act. And given some discussions we had would be indeed something valuable. Note that for the time being, if all your tests inherit from a base class with such TestCleanup method, it would be called always and would help you have this feature already.