Nine tests fails with Python 3.6 but succeed with 3.7 (or is it something about the version of Sphinx)

[mcepl]

When running packaging scripts for openSUSE Tumbleweed (with python3-3.7.3 and python3-Sphinx-2.2.2) the test suite passes without any problem, but when I built it for the bit older stable Leap 15.1 distribution (python3-3.6.5 and python3-Sphinx-2.2.2) I get nine tests failing:

[   32s] =================================== FAILURES ===================================
[   32s] _________________ TestConvert.test_anonymous_reference_option __________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_anonymous_reference_option>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78cd8828>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] _____________________ TestConvert.test_disable_inline_math _____________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_disable_inline_math>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78cd8a58>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] ___________________________ TestConvert.test_dryrun ____________________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_dryrun>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78db7518>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] ___________________________ TestConvert.test_no_file ___________________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_no_file>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78dae668>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] _______________________ TestConvert.test_overwrite_file ________________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_overwrite_file>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78cd8780>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] ______________________ TestConvert.test_overwrite_option _______________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_overwrite_option>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78d92f28>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] _________________________ TestConvert.test_parse_file __________________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_parse_file>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78dbb278>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] ______________________ TestConvert.test_underscore_option ______________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_underscore_option>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78dae358>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] _________________________ TestConvert.test_write_file __________________________
[   32s] 
[   32s] self = <tests.test_cli.TestConvert testMethod=test_write_file>
[   32s] 
[   32s]     def setUp(self):
[   32s]         # reset cli options
[   32s]         options.overwrite = False
[   32s]         options.dry_run = False
[   32s]         options.no_underscore_emphasis = False
[   32s]         options.anonymous_references = False
[   32s]         options.disable_inline_math = False
[   32s]         self._orig_argv = copy(sys.argv)
[   32s]         if path.exists(test_rst):
[   32s]             with open(test_rst) as f:
[   32s] >               self._orig_rst = f.read()
[   32s] 
[   32s] tests/test_cli.py:41: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <encodings.ascii.IncrementalDecoder object at 0x7f2d78dbb320>
[   32s] input = b'\nTitle\n=====\n\nSubTitle\n--------\n\n**content**\n\n\xe3\x82\xb5\xe3\x83\x96\xe3\x82\xbf\xe3\x82\xa4\xe3\x83\x88\xe3\x83\xab\n------------\n\n`A link to GitHub <http://github.com/>`_\n\nThis is :math:`E = mc^2` inline math.\n'
[   32s] final = True
[   32s] 
[   32s]     def decode(self, input, final=False):
[   32s] >       return codecs.ascii_decode(input, self.errors)[0]
[   32s] E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 46: ordinal not in range(128)
[   32s] 
[   32s] /usr/lib64/python3.6/encodings/ascii.py:26: UnicodeDecodeError
[   32s] =============================== warnings summary ===============================
[   32s] tests/test_renderer.py:80
[   32s]   /home/abuild/rpmbuild/BUILD/m2r-0.2.1/tests/test_renderer.py:80: DeprecationWarning: invalid escape sequence \ 
[   32s]     prolog + '\nabc def\ :raw-html-m2r:`<br>`\nghi' + '\n',
[   32s] 
[   32s] tests/test_renderer.py:247
[   32s]   /home/abuild/rpmbuild/BUILD/m2r-0.2.1/tests/test_renderer.py:247: DeprecationWarning: invalid escape sequence \ 
[   32s]     '\na co:\ ``de`` and `RefLink <http://example.com>`_ here.\n',
[   32s] 
[   32s] tests/test_renderer.py:255
[   32s]   /home/abuild/rpmbuild/BUILD/m2r-0.2.1/tests/test_renderer.py:255: DeprecationWarning: invalid escape sequence \ 
[   32s]     '\na `RefLink <http://example.com>`_ and co:\ ``de`` here.\n',
[   32s] 
[   32s] tests/test_renderer.py:707
[   32s]   /home/abuild/rpmbuild/BUILD/m2r-0.2.1/tests/test_renderer.py:707: DeprecationWarning: invalid escape sequence \ 
[   32s]     'This is a\ [#fn-1]_ '
[   32s] 
[   32s] tests/test_renderer.py:708
[   32s]   /home/abuild/rpmbuild/BUILD/m2r-0.2.1/tests/test_renderer.py:708: DeprecationWarning: invalid escape sequence \ 
[   32s]     'footnote\ [#fn-2]_ ref\ [#fn-ref]_ with rst [#a]_.',
[   32s] 
[   32s] -- Docs: https://docs.pytest.org/en/latest/warnings.html
[   32s] ============= 9 failed, 75 passed, 1 skipped, 5 warnings in 0.70s ==============

Complete build log on Leap_15.1

[eterna2]

u need to fix the version of mistune to 0.8.* because they just release v2.

[mcepl]

u need to fix the version of mistune to 0.8.* because they just release v2.

Are you sure it helps? v2 is just a prerelease, I don’t like pushing that into our distribution unless there is some real reason to do so.

[eterna2]

Yeah. I mean u need to make sure u are using 0.8 rather than 2.0.

U can look at #43 for what I meant.

Currently there are no version lock for m2r, this error happens when v2 is install instead of 0.8.

[mcepl]

Currently there are no version lock for m2r, this error happens when v2 is install instead of 0.8.

That is not possible. Take a look at the attached log:

[   25s] python3-mistune-0.8.4-lp151.20.1      ########################################
...
[   26s] python2-mistune-0.8.4-lp151.20.1      ########################################

[eterna2]

Hmmm. Then u probably have a diff error from me.

Cuz I was searching for the same question when my Travis for 3.6 failed while 3.7 passed.

Then I updated my requirements.txt and it worked. But I can't find my logs any more. Too many runs to remember which one was it.

But u probably have a diff issue then?

[mcepl]

In the end it is apparently all about PEP-538, the default encoding with 3.7 is UTF-8, whereas it was US-ASCII with 3.6.

This patch provides fix to this issue.