nose
nose copied to clipboard
nose-devs
--no-skip is broken with Python 3
In my repository https://github.com/mitya57/pymarkups, I have a test defined as.
@unittest.skipUnless(TextileMarkup.available(), 'Textile not available')
class TextileTest(unittest.TestCase):
...
When running it when Textlile is not installed, I get:
$ python3 -m nose --verbose
...
test_mathjax_loading (tests.test_restructuredtext.ReStructuredTextTest) ... ok
test_textile (tests.test_textile.TextileTest) ... SKIP: Textile not available
test_exceptions (tests.test_web.WebTest) ... ok
...
Ran 24 tests in 1.521s
OK (SKIP=1)
But when I add --no-skip to the options, I get:
$ python3 -m nose --verbose --no-skip
...
test_mathjax_loading (tests.test_restructuredtext.ReStructuredTextTest) ... ok
test_textile (tests.test_textile.TextileTest) ... test_exceptions (tests.test_web.WebTest) ... ok
...
Ran 24 tests in 1.515s
OK
Yes, without the newline between the two tests. Yes, in the final message it does not mention that a test was skipped.
The expected result will be that the test fails with a SkipTest exception, and the overall result is OK (FAIL=1).
@jszakmeister Sure:
$ cat test.py
import unittest
class MyTestCase(unittest.TestCase):
@unittest.skip("skipping this one")
def test_foo(self):
pass
$ python3.4 -m nose test.py --verbose
test_foo (test.MyTestCase) ... SKIP: skipping this one
----------------------------------------------------------------------
Ran 1 test in 0.033s
OK (SKIP=1)
$ python3.4 -m nose test.py --verbose --no-skip
test_foo (test.MyTestCase) ...
----------------------------------------------------------------------
Ran 1 test in 0.003s
OK
Actually it seems it's broken with Python 2.7 as well, and maybe even it's the same issue as #512.
The expected result will be that the test fails with a SkipTest exception, and the overall result is OK (FAIL=1).
Why do you say this should be the result? Isn't saying --no-skip the same as saying "please don't add skip support". In which case, SkipTest should be treated like any other exception?
Err, sorry, indeed I meant that it should fail, like with any other exception.
Hmm. I'm not sure what to do here. It used to be that unittest had no concept of skipping tests, and Nose provided this capability. With unittest supporting skipping out-of-the-box, there's no good way to disable it, especially when using unittest.TestCase, as it will go through unittest's mechanisms and it will call addSkip() on the result. At this point, we don't have any exception data, and that makes it impossible to give a good result here. :-( Well, at least without monkey-patching, and I'm not real keen on that.
I can make the error look like this:
:: python selftest.py --no-skip -v -s functional_tests/support/issue889
test_foo (test.MyTestCase) ... ERROR
======================================================================
ERROR: test_foo (test.MyTestCase)
----------------------------------------------------------------------
skipping this one
----------------------------------------------------------------------
Ran 1 test in 0.002s
FAILED (errors=1)
Which isn't very intuitive, IMHO. Another alternative might be to drop --no-skip support, now that the ecosystem has evolved.
Dropping that option will be fine, I think. The fact that is has not been working since Python 2.7 means that nobody relied on it anyway.
So this feature is broken and going to be deprecated? I have an integration-test that takes REALLY long, and also doesn't work when nose runs each method sequentially (I think this has to do with objects not getting garbage collected and the process running out of memory, but I am not sure)... so I have the whole class marked as 'skip', and I wrote a small script to call each test with a subprocess.Popen command like this:
process = subprocess.Popen('cd tests & nosetests --no-skip test_schematics.py:TestSchematics.{}'.format(test_method), shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
but as this ticket indicates, this functionality of running a skipped test is completely broken.
