Error building sphinx documentation with python-3.10

[jorti]

The builds of the python-btrfs v13 package have begun to fail in Fedora rawhide with Python 3.10 with this error:

+ /usr/bin/make -O -j8 V=1 VERBOSE=1 html
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v4.1.2
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 3 source files that are out of date
updating environment: [new config] 3 added, 0 changed, 0 removed
reading sources... [ 33%] btrfs
WARNING: html_static_path entry '_static' does not exist

Exception occurred:
  File "/usr/lib/python3.10/site-packages/sphinx/util/docfields.py", line 369, in transform
    new_list += fieldtype.make_field(fieldtypes, self.directive.domain, items,
TypeError: patched_make_field() got an unexpected keyword argument 'inliner'
The full traceback has been saved in /tmp/sphinx-err-dfd4p4s4.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [Makefile:53: html] Error 2

This is the full sphinx backtrace in the file /tmp/sphinx-err-dfd4p4s4.log:

# Sphinx version: 4.1.2
# Python version: 3.10.0rc1 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 3.0.1
# Last messages:
#   Running Sphinx v4.1.2
#   making output directory...
#   done
#   building [mo]: targets for 0 po files that are out of date
#   building [html]: targets for 3 source files that are out of date
#   updating environment:
#   [new config]
#   3 added, 0 changed, 0 removed
#   reading sources... [ 33%] btrfs
# Loaded extensions:
#   sphinx.ext.mathjax (4.1.2) from /usr/lib/python3.10/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /usr/lib/python3.10/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /usr/lib/python3.10/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /usr/lib/python3.10/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /usr/lib/python3.10/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /usr/lib/python3.10/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /usr/lib/python3.10/site-packages/alabaster/__init__.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /usr/lib/python3.10/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.1.2) from /usr/lib/python3.10/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.1.2) from /usr/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.viewcode (4.1.2) from /usr/lib/python3.10/site-packages/sphinx/ext/viewcode.py
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/usr/lib/python3.10/site-packages/sphinx/application.py", line 343, in build
    self.builder.build_update()
  File "/usr/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 293, in build_update
    self.build(to_build,
  File "/usr/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 307, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 414, in read
    self._read_serial(docnames)
  File "/usr/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 435, in _read_serial
    self.read_doc(docname)
  File "/usr/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 475, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File "/usr/lib/python3.10/site-packages/sphinx/io.py", line 188, in read_doc
    pub.publish()
  File "/usr/lib/python3.10/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/lib/python3.10/site-packages/sphinx/io.py", line 108, in read
    self.parse()
  File "/usr/lib/python3.10/site-packages/docutils/readers/__init__.py", line 77, in parse
    self.parser.parse(self.input, document)
  File "/usr/lib/python3.10/site-packages/sphinx/parsers.py", line 100, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
    return method(self, expmatch)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
    return self.run_directive(
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
    result = directive_instance.run()
  File "/usr/lib/python3.10/site-packages/sphinx/ext/autodoc/directive.py", line 173, in run
    result = parse_generated_content(self.state, params.result, documenter)
  File "/usr/lib/python3.10/site-packages/sphinx/ext/autodoc/directive.py", line 116, in parse_generated_content
    nested_parse_with_titles(state, content, node)
  File "/usr/lib/python3.10/site-packages/sphinx/util/nodes.py", line 335, in nested_parse_with_titles
    return state.nested_parse(content, 0, node, match_titles=1)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2344, in explicit_markup
    self.explicit_list(blank_finish)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2369, in explicit_list
    newline_offset, blank_finish = self.nested_list_parse(
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 318, in nested_list_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/lib/python3.10/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2647, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
    return method(self, expmatch)
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
    return self.run_directive(
  File "/usr/lib/python3.10/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
    result = directive_instance.run()
  File "/usr/lib/python3.10/site-packages/sphinx/domains/__init__.py", line 286, in run
    return super().run()
  File "/usr/lib/python3.10/site-packages/sphinx/directives/__init__.py", line 212, in run
    DocFieldTransformer(self).transform_all(contentnode)
  File "/usr/lib/python3.10/site-packages/sphinx/util/docfields.py", line 251, in transform_all
    self.transform(child)
  File "/usr/lib/python3.10/site-packages/sphinx/util/docfields.py", line 369, in transform
    new_list += fieldtype.make_field(fieldtypes, self.directive.domain, items,
TypeError: patched_make_field() got an unexpected keyword argument 'inliner'

[knorrie]

Hi Jorti. That's inconvenient.

There's a really long existing bug in sphinx that causes it to generate cross references in places where it really should not. https://github.com/sphinx-doc/sphinx/issues/2549

In docs/source/conf.py there's a monkey patch for this, from line 363 and further.

It looks like a fix for this was implemented in v4 https://github.com/sphinx-doc/sphinx/pull/8638/commits

-$ git tag --contains 918086b5590763663c1627578085e528f1358384
v4.0.0
[...]

The function that is replaced while applying the workaround has been changed recently, causing the errors you see.

-$ git tag --contains 4534d2d1a5755c8cbc9ef4327eab7e34a85a7de8
v4.1.0
[...]

So, combining this information, it seems that when the major version number of the sphinx lib being used is >= 4, the monkey patch part should be skipped.

I just pushed a change to the develop branch with a little dirty extra amount of duct tape to do this.

Can you test it?

[jorti]

Hi, I can confirm that the commit 99fe101 fixes the issue. I've applied it on top of the v13 tag and the documentation is built successfully with sphinx 4.1.2

[knorrie]

Ok, closing now, since it's fixed. Thanks.