It doesn't considered how much time was spent on SetupTestData

[dzhuang]

Time spent on SetupTestData is counted into the first test in the testcase. That's not fair to compare.

[SebCorbin]

Agreed, it would be awesome to extract the time spend on setUpTestData and report it too. That way it would indicate the need to transform generated data to a fixture for example

[SebCorbin]

I have modified the TimingSuite to record it

    def run(self, result, debug=False):
        topLevel = False
        if getattr(result, '_testRunEntered', False) is False:
            result._testRunEntered = topLevel = True

        for test in self:
            if result.shouldStop:
                break

            if _isnotsuite(test):
                start_time = _time()
                self._tearDownPreviousClass(test, result)
                self._handleModuleFixture(test, result)
                self._handleClassSetUp(test, result)
                result._previousTestClass = test.__class__

                if (getattr(test.__class__, '_classSetupFailed', False) or
                        getattr(result, '_moduleSetUpFailed', False)):
                    continue
                self.save_test_time(
                    f'{test.__class__.__module__}.{test.__class__.__name__}.setUp '
                    f'({importlib.import_module(test.__class__.__module__).__file__})',
                    _time() - start_time
                )

            start_time = _time()

            if not debug:
                test(result)
            else:
                test.debug()
            self.save_test_time(str(test), _time() - start_time)

        if topLevel:
            self._tearDownPreviousClass(None, result)
            self._handleModuleTearDown(result)
            result._testRunEntered = False

        return result