Rednose and processes do not work togeather

[bartlomiej-kurek]

Hi, I've run into a bizarre issue with rednose, python 3.5.2, Ubuntu 16.04. Problem: when multiple inheritance is used rednose discards the failure. It depends on the order of defining test classes/methods.

The code to reproduce the problem: $ cat test_rednose.py

import unittest


class FrontendCase(unittest.TestCase):
    pass

class BackendCase(unittest.TestCase):
    pass

class BackendUserCase(BackendCase):
    pass

class ExistingBackendUserCase(BackendUserCase):
    pass

class FrontendUserCase(FrontendCase, BackendUserCase):
    pass

class ExistingFrontendUserCase(FrontendUserCase, ExistingBackendUserCase):
    pass

class Test_0(FrontendUserCase):
    def test_dummy(self):
        pass


class Test_1(ExistingFrontendUserCase):
    def test_assertion(self):
        print("Should fail")
        self.assertTrue(False)

$ cat nosetests.cfg

[nosetests]
verbosity = 1
with-doctest = 0
logging-level = DEBUG
logging-format = "%(asctime)s %(name)-12s %(levelname)-8s %(module)s:%(lineno)s %(message)s"
nocapture = 1
nologcapture = 1
rednose = 1
processes = 1

Run it:

$ nosetests -c nosetests.cfg test_rednose.py 
Should fail
.
----------------------------------------------------------------------
Ran 1 test in 0.031s

OK

Now, comment out rednose in nosetests.cfg and run again:

$ grep -v rednose nosetests.cfg > nosetests.without_rednose.cfg
$ nosetests -c nosetests.without_rednose.cfg test_rednose.py 
.F
======================================================================
FAIL: test_assertion (test_rednose.Test_1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_rednose.py", line 30, in test_assertion
    self.assertTrue(False)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 2 tests in 0.020s

FAILED (failures=1)

Voila, finally assertTrue(False) fails as it should.

This issue doesn't seem to be related to "--processes" in nosetests. What is really bizarre is that: a) if we just remove Test_0::test_dummy method (leaving Test_0 class empty), the assertion will properly fail. b) The test method in Test_1 class is actually executed up to that assertion point.

I didn't debug rednose itself.

$ python -V Python 3.5.2 $ uname -rmpov 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 GNU/Linux

[JBKahn]

I'll try to reproduce this later this week using your code and dig into it! Thanks for the bug report, what an odd bug.

[JBKahn]

hmmm it does seem like it works with everything but that library, the test output isn't even colored with processes = 1

[JBKahn]

[nosetests]
rednose = 1
processes = 1

alone breaks it.

[JBKahn]

I will look into it.

[JBKahn]

for example,

(rednose) josephkahn (2017-02-15 18:00:18)>~/dev/rednose (master)$ nosetests --rednose test_files/new_tests.py  --processes=1
.
----------------------------------------------------------------------
Ran 1 test in 0.235s

OK
(rednose) josephkahn (2017-02-15 18:00:31)>~/dev/rednose (master)$ nosetests --rednose test_files/new_tests.py
......

-----------------------------------------------------------------------------
6 tests run in 0.178 seconds (6 tests passed)

(rednose) josephkahn (2017-02-15 18:00:35)>~/dev/rednose (master)$ nosetests test_files/new_tests.py  --processes=1
......
----------------------------------------------------------------------
Ran 6 tests in 0.236s

OK

[JBKahn]

I started taking a look and this is non-trivial so I'm not sure I'll have time to do a crazy deep dive on it, it's an artifact of a really awful plugin api for nose. Colorizing the output shouldn't affect this at all, but it does, in ways I don't entirely understand just yet.

[JBKahn]

I can comment all of rednose, except for

    def prepareTestResult(self, result):  # noqa
        """Required to prevent others from monkey patching the add methods."""
        return result

and it still breaks the multiprocess package but I can't see why that is.

while this works

    def prepareTestResult(self, result):  # noqa
        """Required to prevent others from monkey patching the add methods."""
        return None

[JBKahn]

Fixed in here: https://github.com/JBKahn/rednose/pull/22

[JBKahn]

I'm not sure it actually works, I think it might just be you cannot use them together and it's running only one of the two now.