Do _TestInfo objects really need a persistent unittest.TestResult reference? They are very hard to pickle...

[crswong888]

Reason

My current objective is to subclass unittest.TestCase to hook the run() method such that each test method runs from a multiprocessing.Process spawn on Windows 10. The trickiest part about this is that unittest.TestResult objects have a few unpicklable attributes, i.e., _original_stderr, _original_stdout, and sometimes stream.stream. Luckily, handling this case-by-case is fairly straightforward, especially with unittest.TextTestResult, by temporarily modifying the result object.

When my TestCase.run() hook receives _XMLTestResult objects, things become way messier because the result list attributes, e.g., successes, store references to _TestInfo objects collectively storing references to copies of all former states of the result object. This isn't a problem in the standard xmlrunner framework because all _TestInfo.test_result refer to the same object. However, since I'm pickling and unpickling the result, I end up with multiple, basically dormant copies. Therefore, I would have to loop through all copies to make each one pickleable. I hope this is all making sense so far...

Design

The first solution I came up with was to update the test_result attribute to the original _XMLTestResult object on the parent process each time a _TestInfo reference was appended to one of the many lists. Then, I would only have to modify the result object passed to run() to make everything picklable and not loop through a bunch of old copies. It worked but was ugly, and I didn't want my run() method requiring special treatment for _XMLTestResult objects.

Then, I noticed that the test_result attribute has no further use once test_finished() gets invoked. Thus, I decided that the simplest thing to do was delete test_result after the initial (and only?) invocation of test_finished() by forcibly overriding the class in the same module where my unittest.TestCase subclass is declared:

import xmlrunner

class _TestInfo(xmlrunner.result._TestInfo):
    """
    Instances of this class override the default value for the 'infoclass' attribute of
    xmlrunner.result._XMLTestResult objects.

    Upon pickling and unpickling, the 'test_result' attribute of the base class becomes an
    effectively static reference to an instance of unittest.TestResult (or any of its children),
    making it tricky to pickle again on the next test case. This class is merely a hook for deleting
    that attribute once it is no longer needed.
    """
    def test_finished(self):
        super().test_finished()
        delattr(self, 'test_result')

xmlrunner.result._TestInfo = _TestInfo

Impact

First off, is it safe for me to be deleting the test_result attribute in this way? If so, could we make this the official behavior? Either way, it seems inefficient to be using the result object for this purpose. There should be some alternative approach to confirm that a test has finished and compute the elapsed time. Instances of _TestInfo make only very mild use of their test_result attribute and could almost do away with it after construction.